-- Logs begin at Wed 2022-08-24 16:00:53 UTC, end at Wed 2022-08-24 17:01:02 UTC. -- Aug 24 16:25:12.007493 dv-cinder-779578-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 16:25:12.014874 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 16:25:12 2022] *** Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: nodename: dv-cinder-779578-5-iscsi Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: machine: x86_64 Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: clock source: unix Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: pcre jit disabled Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: detected number of CPU cores: 4 Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: current working directory: / Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: detected binary path: /usr/bin/uwsgi-core Aug 24 16:25:12.017695 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: your processes number limit is 47680 Aug 24 16:25:12.018258 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: your memory page size is 4096 bytes Aug 24 16:25:12.018258 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: detected max file descriptor number: 2048 Aug 24 16:25:12.018258 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: lock engine: pthread robust mutexes Aug 24 16:25:12.018258 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: thunder lock: enabled Aug 24 16:25:12.018258 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 16:25:12.018258 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 16:25:12.057525 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: Python main interpreter initialized at 0x55d087fc89c0 Aug 24 16:25:12.057525 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: python threads support enabled Aug 24 16:25:12.057525 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: your server socket listen backlog is limited to 100 connections Aug 24 16:25:12.057525 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: your mercy for graceful operations on workers is 90 seconds Aug 24 16:25:12.057885 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 16:25:12.057963 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: *** Operational MODE: preforking *** Aug 24 16:25:12.058264 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: *** uWSGI is running in multiple interpreter mode *** Aug 24 16:25:12.058264 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: spawned uWSGI master process (pid: 81676) Aug 24 16:25:12.058449 dv-cinder-779578-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 16:25:12.059129 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: spawned uWSGI worker 1 (pid: 81687, cores: 1) Aug 24 16:25:12.059994 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: spawned uWSGI worker 2 (pid: 81688, cores: 1) Aug 24 16:25:12.062211 dv-cinder-779578-5-iscsi devstack@keystone.service[81676]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 16:25:16.363283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:25:16.363283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: from cryptography.utils import int_from_bytes Aug 24 16:25:16.364238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:25:16.364238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: from cryptography.utils import int_from_bytes Aug 24 16:25:16.365022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:25:16.365022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: from cryptography.utils import int_from_bytes Aug 24 16:25:16.365161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:25:16.365161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: from cryptography.utils import int_from_bytes Aug 24 16:25:16.416320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 16:25:16.416524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 16:25:16.416694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] command line args: [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 16:25:16.416966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 16:25:16.417146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ================================================================================ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 16:25:16.417392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] admin_token = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.417882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config_dir = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.418072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.418262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config_source = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.418434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] debug = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.418634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_config_append = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_dir = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.420538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 16:25:16.421139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 16:25:16.421204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] command line args: [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 16:25:16.421322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 16:25:16.421444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ================================================================================ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 16:25:16.421570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] admin_token = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.421953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config_dir = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config_source = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] debug = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.422909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.423935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_config_append = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_dir = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] publish_errors = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.424927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.425984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.426053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.426112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] standard_threads = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.426297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.426894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_journal = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_json = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.427794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_stderr = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_syslog = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.428547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.428767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.428871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.429083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.429235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.429322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.429539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.429638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.429761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] publish_errors = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.429840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.430042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.430129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.430262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.430371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.430479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.430576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.430730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.430854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.431008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.431190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.431298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.431376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] standard_threads = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.431455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.431586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.431715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.431888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.432011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.432150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.432291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.432403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_journal = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.432522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.432610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_json = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.432689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.432810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_stderr = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.432911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.432992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_syslog = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.433142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.433241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:25:16.433367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.433543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.433661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.433859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.433973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.434997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.435871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.436039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.436366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.436554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.436739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.436900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.437956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.438915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.439916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.440982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.441127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.441284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.441368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.441519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.441652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.441895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.442125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.442845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.442964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.443938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.444955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.445969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.446979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.447799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.448987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.449942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.450129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.450216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.450403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.450491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.450829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.451913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.452901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.external = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.453988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.password = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.token = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.454995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.455868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.456845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.457919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.458853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.459939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.460967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.461899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.external = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.462879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.password = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.463852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.token = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.464967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.465915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.466953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.467930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.468973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.469999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.470964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.471993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.472865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.473939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.474875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.475882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.476964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.477936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.478944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.479908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480159 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.480870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.481946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.482969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.483910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.484873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.485993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.486978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.487851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.488733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.489363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.489501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.489982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.490971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.491921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.492978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.493966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.494923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.495909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.496938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.497930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.498891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.499834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.500909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.501869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.502977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.503912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.504010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.504172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.504252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.504421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.504565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.505291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.505739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.505818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.505891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.505963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 16:25:16.506394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.506812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.507065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.507310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.507544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.507807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.508135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.508378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.508613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.508848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.509081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.509323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.509570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.509796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.510036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.510328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.510587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.510828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.511073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.511308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.511545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.511778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.512035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.512271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.512504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.512745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.512977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.513219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.513450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.513695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.513928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.514168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.514394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.514630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.514861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:25:16.515029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 16:25:16.535280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.535511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.535765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.539939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.540220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.540412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.544493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.549182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.573874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.574130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.578069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.578301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.627716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.628240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.677901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.678133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.678585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.678819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.836539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.836768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.836982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.837991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.838204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.838415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.860510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.860779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.860884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.861970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.862111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.873108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.873277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.886597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.886731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.886817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.886919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.886994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.887856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.888006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:25:16.904916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.905125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:25:16.905470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.905584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.905667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:25:16.905792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:25:16.905936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.906130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.906212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.906327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:25:16.906601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.906683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.906862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.906952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.907361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.907621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.916327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.916607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.916883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.917070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.917154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.917348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.917874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.918167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.924351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.925163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.927406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.927688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.927991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.928166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.928277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.928452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.928550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.928736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.928820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929555 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.929989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.930094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.930264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.930474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.930747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.931016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.953232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.953755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.954684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.954970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.955195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.955480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.960323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.960610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.960742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.960898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.960997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.961266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.967245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.967512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.967597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.967797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.971594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.971826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.971910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.972983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.973075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.973283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.973707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.973791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.973866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.973940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.990760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.990861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.991033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.991117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.994862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.994972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:16.998057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.998149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.998364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:16.998450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.002521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.002635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.006149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.006470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.009362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.009665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.009759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.009952 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.010906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.011120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.027301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.027445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.027599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.027736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.027869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.028044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.028156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.028313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.034905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.035121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.036540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.036697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.036817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.036993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.039426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.039658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.039754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.039963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.040049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.040264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.040351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.040552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.040635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.040828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.052526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.052817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.052991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.053101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.053278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.053381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.053562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.053656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.053846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.053939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.054130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.054213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.054410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.054725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.069936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.070618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.072929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.073638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.075759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.076543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.077215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.077983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.078665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.078954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.079243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.079454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.079750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.080068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.086228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.087132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.087663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.088607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.090548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.090840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.091125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.091408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.091512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.091780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.092094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.092386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.100045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.100338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.100618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.101068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.101364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.101668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.107382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.107668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.107963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.108241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.108522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.108604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:25:17.108795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.108881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.109990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.110965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.111049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.111265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.111539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.111814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.112099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:25:17.146334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d087fc89c0 pid: 81687 (default app) Aug 24 16:25:17.146442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d087fc89c0 pid: 81688 (default app) Aug 24 16:25:17.149748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cc245-13d5-4227-b644-e848f2330d94 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.150035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cc245-13d5-4227-b644-e848f2330d94 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.150310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cc245-13d5-4227-b644-e848f2330d94 None None] PATH_INFO: `/v3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.154468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:23.770446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.771050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.771050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.775675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:23.782119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.782549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.782664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.852582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: INFO dbcounter [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] Registered counter for database keystone Aug 24 16:25:23.873502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG oslo_db.sqlalchemy.engines [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81688) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 16:25:23.884123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writer thread running {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 16:25:24.064877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG passlib.handlers.bcrypt [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81688) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 16:25:24.065298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG passlib.handlers.bcrypt [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81688) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 16:25:24.120149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:24.121727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:24.251866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 16:25:24.670075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.738636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: INFO dbcounter [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] Registered counter for database keystone Aug 24 16:25:24.760352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG oslo_db.sqlalchemy.engines [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81687) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 16:25:24.766550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writer thread running {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 16:25:24.888942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5cb04-7231-4acb-9098-b2607fc617de None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.893425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.894057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.894336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.894613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.903501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.954750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 16:25:24.955038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c7704-6fee-429e-883d-aa3f209b73e0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.965277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:25:24.971179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.985446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c35ec-41e7-4a85-8d07-afefdc0f0a9e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.989820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.990951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.991236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.991509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] PATH_INFO: `/v3/projects/ad742c0425e4471ab69af90dda50deea` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.993096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=ad742c0425e4471ab69af90dda50deea)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:25.045999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 16:25:25.046344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f77f6e-1f2d-456d-87eb-81abceded3f0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:25.050891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/projects/ad742c0425e4471ab69af90dda50deea => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:26.998703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf26be-784d-4156-9e0f-2af038bd85d4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:26.999231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf26be-784d-4156-9e0f-2af038bd85d4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:26.999231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf26be-784d-4156-9e0f-2af038bd85d4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.003343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:27.009557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.009986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.010120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.030542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:27.032351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:27.040777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 16:25:27.315774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf26be-784d-4156-9e0f-2af038bd85d4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:27.328572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf26be-784d-4156-9e0f-2af038bd85d4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:27.332780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e850a98-d23e-46c2-8770-68b0a3f8e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:27.333381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850a98-d23e-46c2-8770-68b0a3f8e5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.333668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850a98-d23e-46c2-8770-68b0a3f8e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.333944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850a98-d23e-46c2-8770-68b0a3f8e5d7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.342323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e850a98-d23e-46c2-8770-68b0a3f8e5d7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:27.344133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e850a98-d23e-46c2-8770-68b0a3f8e5d7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:27.353057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:25:27.358445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:27.367746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3983f-e157-4214-b2a8-40633ca31a91 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:27.372440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9363c9-16c7-47fa-8a81-4acb684e545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:27.373069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9363c9-16c7-47fa-8a81-4acb684e545a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.373353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9363c9-16c7-47fa-8a81-4acb684e545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.373627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9363c9-16c7-47fa-8a81-4acb684e545a admin admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.374936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9363c9-16c7-47fa-8a81-4acb684e545a admin admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:27.376804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9363c9-16c7-47fa-8a81-4acb684e545a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:27.378336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:29.389507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087d8cc-1c62-45db-9fda-3b795150d94f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.390049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087d8cc-1c62-45db-9fda-3b795150d94f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.390049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087d8cc-1c62-45db-9fda-3b795150d94f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.391401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:29.396672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.397214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.397214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.409566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a547e-4719-4905-b9ed-15819511ab3a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.409879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a547e-4719-4905-b9ed-15819511ab3a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.410154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a547e-4719-4905-b9ed-15819511ab3a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.411457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:29.416721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:29.417282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.417568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.418113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.418220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:29.460464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG passlib.handlers.bcrypt [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81687) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 16:25:29.460879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG passlib.handlers.bcrypt [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81687) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 16:25:29.498841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:29.508149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:29.509759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:29.614219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:29.769672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:29.794280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:29.879399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924a1a0-9086-4855-99cc-932b3cc95146 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:29.882363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:29.915995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:29.916639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.916918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.917201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.928734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:29.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:29.938143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:29.938981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:29.944061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:29.988195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73672b57-fed6-47c7-b863-dbdf737c31e2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:29.990658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.025883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.026664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.026953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.027241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.033388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.046720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2d485-3fd2-4b8a-a682-eebcc19b0948 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.081891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.082660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.082949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.083321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.084646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:30.085126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.086543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.097420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.098371 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:30.098949 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.104541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:30.197466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:25:30.203078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:30.204117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327ad64-291c-427b-b4f2-cc85c41fc03b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.205740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.238063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.238699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.238976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.239312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.241030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.242485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.244161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:30 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:30.249624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:30.264658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.306206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be85e-1ef6-4a07-bd31-d7770cd5264b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.341909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.342768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.343064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.343325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.351126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3232454-7d1e-4f83-852c-4db5f959a731 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.361462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.362940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:30.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:30.382639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:30.383412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:30.383412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:30.383412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:25:30.383412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:30.385529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.386164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.386442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.386714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.387829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.388678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:30.389324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.390978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:30.396477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:30.446287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.481345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.491238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922438a7-16e3-4bc2-94fb-964c49ff25ef None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.500992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161536b2-5e0a-475c-a0db-97d678d6669a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.525059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eca0f-c629-4064-8893-390360808109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.525856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eca0f-c629-4064-8893-390360808109 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.526143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eca0f-c629-4064-8893-390360808109 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.526419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eca0f-c629-4064-8893-390360808109 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.527061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eca0f-c629-4064-8893-390360808109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:30.527365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eca0f-c629-4064-8893-390360808109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.528801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eca0f-c629-4064-8893-390360808109 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.534195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.534797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.535076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.535346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.536106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.domain.id=default, user.id=bcc81a9e50c0416682e495a7f72fd265) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:30.536411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.538429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392ea27-b565-466b-974c-28b6c7c9764c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.545474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Aug 24 16:25:30 2022] GET /identity/v3/role_assignments?user.id=bcc81a9e50c0416682e495a7f72fd265&scope.domain.id=default&role.id=580c9e29995a41c49a5a943d7b48532c => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:30.548121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:32.770909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1b22-c0ae-4c03-bccd-9b3d7f6e8ff8 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.772353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1b22-c0ae-4c03-bccd-9b3d7f6e8ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.772799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1b22-c0ae-4c03-bccd-9b3d7f6e8ff8 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.774610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:32.785865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10cc42-35ff-4743-832a-da42162a9a80 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.786680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10cc42-35ff-4743-832a-da42162a9a80 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.787099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10cc42-35ff-4743-832a-da42162a9a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.810814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8c10cc42-35ff-4743-832a-da42162a9a80 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:32.812325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8c10cc42-35ff-4743-832a-da42162a9a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:32.897930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:33.120518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa2fb1-c781-4347-b37c-f76e5b85122d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.121142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa2fb1-c781-4347-b37c-f76e5b85122d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.121641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa2fb1-c781-4347-b37c-f76e5b85122d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.123196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:33.128650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb0f4-f7d9-406c-8587-221503e6f4b9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.129152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb0f4-f7d9-406c-8587-221503e6f4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.129644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb0f4-f7d9-406c-8587-221503e6f4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.158506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6238a15-061a-4f0a-9fd6-34e9ba00a566 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.159104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6238a15-061a-4f0a-9fd6-34e9ba00a566 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.159600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6238a15-061a-4f0a-9fd6-34e9ba00a566 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.161161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 14/25] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:33.162670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1a8eb0f4-f7d9-406c-8587-221503e6f4b9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:33.164134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1a8eb0f4-f7d9-406c-8587-221503e6f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:33.172229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.172777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.173245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:33.208673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:33.275826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.313554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 12/26] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:33.314572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4917e4f-5b97-4171-b8ff-4981a66c17a2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.314861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4917e4f-5b97-4171-b8ff-4981a66c17a2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.315139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4917e4f-5b97-4171-b8ff-4981a66c17a2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.316480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:33.322804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.323335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.323835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.352973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 15/28] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 16:25:33.360958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:33.361082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.362154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:33.500364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.539520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af073b4-08a1-4751-99cc-24afd70de053 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.570038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 14/29] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:33.592403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.593044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.593320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.593592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.608980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.610447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:33.631562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:33.632747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:33.633566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:33.633566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.633566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:33.633566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 16/30] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:33.638628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.638750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.746873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437197a-783d-4d1c-aeb7-6b5dda75cf45 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.782325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.782959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.785502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.785502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.785502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:33.785502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.785785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.826285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 17/31] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:33.828328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.829014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.847898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56fee6-5593-4dd5-af8c-02403ebe9908 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.860282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.894755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.897202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.899184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.899615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.900496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.902761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.983422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff06f50-f6f7-4688-a20c-105e600d9ec8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.023792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.024709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.025161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.025622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.037952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:34.043651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 15/32] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 16:25:33 2022] POST /identity/v3/roles => generated 219 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:34.043874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:34.045027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:34.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:34.047914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:34.048873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.049151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.114437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.222879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f71a9-ceb8-43e2-a267-2963153b2ee3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.277822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b907c39-7360-49ef-a038-61eee838dc71 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.282620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315003b-7121-4f2d-9272-89fb8a30ed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.283231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315003b-7121-4f2d-9272-89fb8a30ed77 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.283518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315003b-7121-4f2d-9272-89fb8a30ed77 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.283796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315003b-7121-4f2d-9272-89fb8a30ed77 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.284515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315003b-7121-4f2d-9272-89fb8a30ed77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.286604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315003b-7121-4f2d-9272-89fb8a30ed77 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.321884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.323008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.323550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.324079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.325323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.327291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.371396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 16/34] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:25:34 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:34.373396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.399644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.441728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 19/35] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 16:25:34 2022] POST /identity/v3/roles => generated 225 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:34.443242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cc56c-c611-434b-bad7-38db30a690ba None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.444573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cc56c-c611-434b-bad7-38db30a690ba None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.444860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cc56c-c611-434b-bad7-38db30a690ba None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.446125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 20/36] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:34.457528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.457838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.458092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.501972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:34.504669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:34.554742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083cea-bae4-4f20-8974-bfb822e6c635 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.597881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.598513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.598797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.600403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.600403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:34.601873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.603228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21adc3-40cf-495b-9a09-c0f013f92bb8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.618885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 17/37] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.621299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac74d5c-d6b1-4bb5-baf0-cbd5ceed18a2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.622061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac74d5c-d6b1-4bb5-baf0-cbd5ceed18a2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.623136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac74d5c-d6b1-4bb5-baf0-cbd5ceed18a2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.625145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 21/38] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:34.625145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.625794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 18/39] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:34.637937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.638235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.638504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.673859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:34.674442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:34.679901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.759157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6c7f7-76f0-4593-a81a-7b3043e52479 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.761544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 19/40] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:34.800985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70441dab-d543-4b64-a509-73d257475428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.801777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70441dab-d543-4b64-a509-73d257475428 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.802068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70441dab-d543-4b64-a509-73d257475428 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.802346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70441dab-d543-4b64-a509-73d257475428 None admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.804228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70441dab-d543-4b64-a509-73d257475428 None admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.806219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70441dab-d543-4b64-a509-73d257475428 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.808431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 22/41] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.813469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.826950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.916407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70441dab-d543-4b64-a509-73d257475428 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.917158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30080a4-eb23-4e64-808a-d7c14e78ad6a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.955141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.956078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.956374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.956657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.958034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.960848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.962464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 20/42] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.970706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.047066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70441dab-d543-4b64-a509-73d257475428 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.049382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.063615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.082058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.082672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.082955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.083231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.083899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.085514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.086281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cc88c-99e5-4f27-9c5d-0ebc8d72001c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.104448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 23/43] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 16:25:34 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:25:35.104448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.128055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.128881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.129165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.129443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] PATH_INFO: `/v3/domains/default/users/bcc81a9e50c0416682e495a7f72fd265/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.132364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bcc81a9e50c0416682e495a7f72fd265, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.134059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.151207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 21/44] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Aug 24 16:25:34 2022] PUT /identity/v3/domains/default/users/bcc81a9e50c0416682e495a7f72fd265/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:25:35.337434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e187d-dd36-4d88-b9bd-2cab3df4780c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.371671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3ed49-d08b-4a91-a377-0d2b5cdda194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.372338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3ed49-d08b-4a91-a377-0d2b5cdda194 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.372623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3ed49-d08b-4a91-a377-0d2b5cdda194 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.372900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3ed49-d08b-4a91-a377-0d2b5cdda194 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.374018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3ed49-d08b-4a91-a377-0d2b5cdda194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.375464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3ed49-d08b-4a91-a377-0d2b5cdda194 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.377080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 24/45] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:35.383368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.483957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bc059-6022-480a-ae34-e91065ff527b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.518044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539226-a22c-4dfe-85e4-0ee8f8dc87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.518655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539226-a22c-4dfe-85e4-0ee8f8dc87be None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.518939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539226-a22c-4dfe-85e4-0ee8f8dc87be None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.519214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539226-a22c-4dfe-85e4-0ee8f8dc87be None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.519871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6539226-a22c-4dfe-85e4-0ee8f8dc87be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.521348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6539226-a22c-4dfe-85e4-0ee8f8dc87be None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.531112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.549334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 22/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:25:35 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:25:37.087039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065918-37d6-4d2f-ae87-2e4821394e10 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.087537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065918-37d6-4d2f-ae87-2e4821394e10 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.087621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065918-37d6-4d2f-ae87-2e4821394e10 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.088996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 25/47] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:37.094273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa90139-6d56-4d6c-a8db-555b8dfa2136 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.094648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa90139-6d56-4d6c-a8db-555b8dfa2136 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.094843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa90139-6d56-4d6c-a8db-555b8dfa2136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.113054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4fa90139-6d56-4d6c-a8db-555b8dfa2136 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:37.114245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4fa90139-6d56-4d6c-a8db-555b8dfa2136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:37.181400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 23/48] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:37.450160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065918-37d6-4d2f-ae87-2e4821394e10 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.451210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3d6a-81b7-4644-8cd8-867df0b4b009 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.451506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3d6a-81b7-4644-8cd8-867df0b4b009 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.451789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3d6a-81b7-4644-8cd8-867df0b4b009 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.453209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 24/49] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:37.458218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.458505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.458785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.477727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:37.478926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:37.483022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.547550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 25/50] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:37.555228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065918-37d6-4d2f-ae87-2e4821394e10 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.589572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.590205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.590488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.590765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.602195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:37.606975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:37.607874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:37.608714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:37.615275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.665759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.717438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91280-7935-40f0-9a2d-de79777b1b74 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.751347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.751981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.752268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.752547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.753218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:37.753529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:37.754915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:37.763317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 26/52] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:37.768851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.814823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.862090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.872480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3d43e-023d-48b3-840c-bdf29ff132f1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.905722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.906349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.906627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.906901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.908051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:37.909489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:37.911046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:37.916710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ade62-8626-4594-a728-8f9204cb5aeb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.917080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.930787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.952333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.952978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.953259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.953545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.954672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:37.956150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:37.957757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 27/54] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:37.963550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.021220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8954e-263d-4f31-94b6-454bf94068e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.059812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.060566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.060945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.061286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.062497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.064003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.065547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 28/55] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:38.068079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a39f-3816-47eb-b6ec-d14ff98c1dc9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.072015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.101013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:38.104650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476beb09-dbd0-4e97-b098-28e58b2d842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.105265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476beb09-dbd0-4e97-b098-28e58b2d842a None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.105547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476beb09-dbd0-4e97-b098-28e58b2d842a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.105822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476beb09-dbd0-4e97-b098-28e58b2d842a None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.106469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476beb09-dbd0-4e97-b098-28e58b2d842a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.107888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476beb09-dbd0-4e97-b098-28e58b2d842a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.136815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 28/56] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Aug 24 16:25:37 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:25:38.179246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f454-6bfc-4af9-ad03-adad7721a0bf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.216561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.217194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.217479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.217760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.218522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.domain.id=default, user.id=bcc81a9e50c0416682e495a7f72fd265) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:38.218841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.220376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b9bd-6a54-4b24-a423-c9845b5bf8c6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.227659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 29/57] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Aug 24 16:25:38 2022] GET /identity/v3/role_assignments?user.id=bcc81a9e50c0416682e495a7f72fd265&scope.domain.id=default&role.id=580c9e29995a41c49a5a943d7b48532c => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:40.064579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde023b-9484-4b29-8aaf-2ad3020b2451 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.065112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde023b-9484-4b29-8aaf-2ad3020b2451 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.065205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde023b-9484-4b29-8aaf-2ad3020b2451 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.066499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 29/58] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:40.072836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.073226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.073393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.094780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:40.096013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:40.165427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 30/59] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:40.439049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde023b-9484-4b29-8aaf-2ad3020b2451 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.469935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.541319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde023b-9484-4b29-8aaf-2ad3020b2451 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.574220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7afc0-570a-4c3a-8157-1abd30d0a4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.574849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7afc0-570a-4c3a-8157-1abd30d0a4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.575135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7afc0-570a-4c3a-8157-1abd30d0a4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.575413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7afc0-570a-4c3a-8157-1abd30d0a4d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.576565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7afc0-570a-4c3a-8157-1abd30d0a4d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.577997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7afc0-570a-4c3a-8157-1abd30d0a4d9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.579553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 30/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.585764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.609136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.682683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f061fb-80f9-4acf-8550-edb21861ff84 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.714445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1e77e-a13a-4cb6-aca9-cb3f851e85dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.714994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1e77e-a13a-4cb6-aca9-cb3f851e85dd None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.715271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1e77e-a13a-4cb6-aca9-cb3f851e85dd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.715543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1e77e-a13a-4cb6-aca9-cb3f851e85dd None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.716204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1e77e-a13a-4cb6-aca9-cb3f851e85dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.717626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1e77e-a13a-4cb6-aca9-cb3f851e85dd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.743178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 31/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:40 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:43.666742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346307c-b752-4c0a-99da-a04327f5d6a3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.667249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346307c-b752-4c0a-99da-a04327f5d6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.667321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346307c-b752-4c0a-99da-a04327f5d6a3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.668914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 31/62] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:43.676495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d5af8-5da6-47b7-8a6c-41c338633e19 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.676904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d5af8-5da6-47b7-8a6c-41c338633e19 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.677124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d5af8-5da6-47b7-8a6c-41c338633e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.706532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6e7d5af8-5da6-47b7-8a6c-41c338633e19 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:43.707732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6e7d5af8-5da6-47b7-8a6c-41c338633e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:43.797172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68658a7f-6c60-4996-bae5-e0b7e6f1f382 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.797542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68658a7f-6c60-4996-bae5-e0b7e6f1f382 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.797726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68658a7f-6c60-4996-bae5-e0b7e6f1f382 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.799238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 32/63] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:43.805003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff36c88-131b-415c-b044-39c3ba6f0e24 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.805288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff36c88-131b-415c-b044-39c3ba6f0e24 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.805567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff36c88-131b-415c-b044-39c3ba6f0e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.817232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 32/64] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:43.833318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-aff36c88-131b-415c-b044-39c3ba6f0e24 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:43.834624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-aff36c88-131b-415c-b044-39c3ba6f0e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:43.917603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 33/65] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:44.138427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e946ad-5fa7-4384-82e9-f8a0d0551154 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.138745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e946ad-5fa7-4384-82e9-f8a0d0551154 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.138981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e946ad-5fa7-4384-82e9-f8a0d0551154 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.140888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 33/66] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:44.148717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.149263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.149727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.160804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e946ad-5fa7-4384-82e9-f8a0d0551154 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.180706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.181979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:44.183182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:44.237428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.319997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e946ad-5fa7-4384-82e9-f8a0d0551154 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.333279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 34/67] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:44.351463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.366266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.367090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.367442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.367719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.389713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:44.394579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:44.395753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:44.397298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 34/68] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:44.404958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.472713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9a6c3-a363-4d36-9d06-533bfbf9ece9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.498563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.511605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025f26d-5860-4e08-8d3c-479c3229b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.512289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025f26d-5860-4e08-8d3c-479c3229b003 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.512754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025f26d-5860-4e08-8d3c-479c3229b003 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.513163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025f26d-5860-4e08-8d3c-479c3229b003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.525618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025f26d-5860-4e08-8d3c-479c3229b003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9025f26d-5860-4e08-8d3c-479c3229b003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:44.532667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:44.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:25:44.534657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 35/69] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:44.535442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002346f4-23fc-43fe-a5e4-949a6d954864 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.535649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002346f4-23fc-43fe-a5e4-949a6d954864 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.535932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002346f4-23fc-43fe-a5e4-949a6d954864 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.538000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 36/70] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:44.539032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f375c-5469-445c-a461-b9539d78a22b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.539326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f375c-5469-445c-a461-b9539d78a22b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.539577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f375c-5469-445c-a461-b9539d78a22b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.540846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 37/71] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:44.541530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06ecfb-0bb7-4e9c-8504-22e5ce44b7e8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.543649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd747e-efd9-4e97-87cd-2d89cbf42a66 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.544148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd747e-efd9-4e97-87cd-2d89cbf42a66 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.544611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd747e-efd9-4e97-87cd-2d89cbf42a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.563494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b7bd747e-efd9-4e97-87cd-2d89cbf42a66 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:44.564735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b7bd747e-efd9-4e97-87cd-2d89cbf42a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:44.578710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.579577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.579860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.579972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.580163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.580883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:44.581352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.582876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.595769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 35/72] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:44.595769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.644646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 38/73] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:44.646277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.646627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.646898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.668589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:44.669894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:44.708639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e6499-4d6f-4e4d-b677-5db35ec44f48 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.738405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.744850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 39/74] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:44.745396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.746114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.746226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.746496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.746877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.747588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:44.747897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.749319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.762074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 36/75] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:44.763112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.840584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.862407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ad25c-c570-449e-8de1-f755bab3574d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.878950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a9f9e-98de-4dc3-a510-8e19fe671b76 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.899254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.900150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.900577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.901029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.915721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.917115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.917412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.917806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.918429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.919849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.921558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 40/76] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:44.922394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.930645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:44.935976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:44.936975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:44.937895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:44.939764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 37/77] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:44.940712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.999044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.038197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc0054-e2e7-41ea-bbec-39bc99385ace None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.052038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b07b1c-9bd8-4b2f-914c-50f3ad338a89 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.072471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.073113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.073399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.073682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.074831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.076313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.077916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.080837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.087890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.088544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.088826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.089101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.089919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.100985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:45.105722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:45.106565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:45.107313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:45.107976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.182839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f031522-2e4c-405e-9bab-33750a17811c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.207134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf59f6-2b84-44c7-8fda-9d55f9a1df5c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.217446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.218057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.218338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.218614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.220554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.242469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.243075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.243358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.243635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.244328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:45.244632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.246033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.253953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 39/80] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.256277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.289782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.291265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.359416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbff24c-2590-4216-beeb-9ce055aeb40c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.394730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.395390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.395749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.396076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.398110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.399601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.401255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 40/81] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.402041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.404476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.447144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.497132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 42/82] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:45.497775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.509794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5b19-3799-460a-bf3b-bfc2b5471700 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.543744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.544380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.544662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.544938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.594762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f707-6109-4306-8ea8-a214332f3284 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.602325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.614535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.616121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.629851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.630467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.630766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.631064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.631818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:45.632153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.633514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.641303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 43/83] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.643180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.658012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.716186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 41/84] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:45.717002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.744660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260031b5-d08f-414b-be0c-dbe50bb79209 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.777989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.778536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.778822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.779098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.780254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.781704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.783268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 44/85] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.788886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.816792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea1ca6-9ff1-47f2-825e-0bfed891935d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.823593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.851139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.851756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.852090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.852358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.853007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.854482 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.871045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.883682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 42/86] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:45 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:45.883682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.900725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdd4f4-6039-4dc6-b0dc-d7e9fedd0896 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.941947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.942628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.942900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.943184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.944793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.946275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.948218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 45/87] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.948842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.997182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedfe58-b721-4924-8de5-e7b7e7e0502b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.044264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.044926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.045808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.045808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.047266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.049340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.050942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 43/88] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.051791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.068279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e29617-5e78-4f37-bc85-f11414b884db None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.071467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.111629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.112357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.112639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.112912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.114666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.116351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.118724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 46/89] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.118840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.139291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.182923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7f14-8510-4639-be34-835a37ce15e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.225775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.226396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.226683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.226963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.262518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430316c-a7d0-4517-82bd-66640a06dc69 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.307702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.308306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.308596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.309128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.309912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=7db0d9ecb6fa4688a758312e6c7cd99a) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:46.310801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.312385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.312600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.313861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.322096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 47/90] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/role_assignments?user.id=7db0d9ecb6fa4688a758312e6c7cd99a&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.322747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.390151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.463483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.526247 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 44/91] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:46.527118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.534082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c145a-d51c-4f8e-a80d-7cda0eb22575 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.636910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f1676-59aa-4935-96b4-d2ce0ce62074 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.648683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.649482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.649604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.649864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.674938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.675898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.676411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.676909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.678408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=bcc81a9e50c0416682e495a7f72fd265) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:46.678921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.680620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.686837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 45/92] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/role_assignments?user.id=bcc81a9e50c0416682e495a7f72fd265&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.688059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.734352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.800488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575732e-24f7-420b-a2d6-98c748c83d1c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.837615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.838657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.839171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.839646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.841128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.842846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.845213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 46/93] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.854730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.862167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.863668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.940047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.057304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.062298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 48/94] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:47.064298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12958302-a094-4a7a-9c70-52f398ec16b0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.068385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.137758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.138382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.138659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.138929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.139672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] RBAC: Adding query filter params (role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=7db0d9ecb6fa4688a758312e6c7cd99a) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:47.139997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.141441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.156172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 47/95] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/role_assignments?user.id=7db0d9ecb6fa4688a758312e6c7cd99a&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=87236176cec04669b7f8d8199ce12228 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:47.229691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b0a79-932d-46dd-b608-c077a1c2447f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.269338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff7abc-0a4f-43d2-b07b-e0e1fa78ae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.269992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff7abc-0a4f-43d2-b07b-e0e1fa78ae45 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.270273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff7abc-0a4f-43d2-b07b-e0e1fa78ae45 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.270549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff7abc-0a4f-43d2-b07b-e0e1fa78ae45 None admin] PATH_INFO: `/v3/projects/168ba65d920d4d1da42783d12e1e142b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.271746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff7abc-0a4f-43d2-b07b-e0e1fa78ae45 None admin] RBAC: Authorizing `identity:get_project(project_id=168ba65d920d4d1da42783d12e1e142b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.273240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff7abc-0a4f-43d2-b07b-e0e1fa78ae45 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.274935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 49/96] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/projects/168ba65d920d4d1da42783d12e1e142b => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:47.280793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.401070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b90d10-0751-440f-a224-0f5e30621018 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.444802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.445438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.445767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.446043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.446793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=168ba65d920d4d1da42783d12e1e142b, user.id=7db0d9ecb6fa4688a758312e6c7cd99a) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:47.447100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.448577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae3cc-bd40-42a7-b2e0-17f0b6b65a6c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.454486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 48/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/role_assignments?user.id=7db0d9ecb6fa4688a758312e6c7cd99a&scope.project.id=168ba65d920d4d1da42783d12e1e142b&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.444630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe93d4-c560-4c4b-ac5f-ea889e36a3c1 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.445142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe93d4-c560-4c4b-ac5f-ea889e36a3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.445216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe93d4-c560-4c4b-ac5f-ea889e36a3c1 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.446835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 50/98] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:48.452392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161257c-ebb1-4733-949e-546d43eb0b60 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.452903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161257c-ebb1-4733-949e-546d43eb0b60 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.452981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161257c-ebb1-4733-949e-546d43eb0b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.480989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5161257c-ebb1-4733-949e-546d43eb0b60 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:48.482274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5161257c-ebb1-4733-949e-546d43eb0b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:48.508035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.571044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 49/99] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:48.628046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54e02-8906-425a-9259-c5856ae09d5d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.628342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54e02-8906-425a-9259-c5856ae09d5d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.628615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d54e02-8906-425a-9259-c5856ae09d5d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.629986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 51/100] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:48.636142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3242a-b5aa-45d5-ad2f-e1737374c10d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.636420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3242a-b5aa-45d5-ad2f-e1737374c10d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.636701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3242a-b5aa-45d5-ad2f-e1737374c10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.656091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7aa3242a-b5aa-45d5-ad2f-e1737374c10d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:48.657289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7aa3242a-b5aa-45d5-ad2f-e1737374c10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:48.750559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 50/101] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:48.929652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4340cf5-d08d-47df-bdb6-63529d564a63 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.929964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4340cf5-d08d-47df-bdb6-63529d564a63 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.930215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4340cf5-d08d-47df-bdb6-63529d564a63 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.931646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 52/102] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:48.950007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.950296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.950572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.976918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:48.978148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:49.079481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4340cf5-d08d-47df-bdb6-63529d564a63 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.079694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 51/103] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:49.083367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.111631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.132453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.226827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401996aa-1be9-4c80-82a9-940122a90ae7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.266920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.268220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.268723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.269199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.280991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:49.286115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:49.286977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:49.287773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:49.290489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 52/104] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:49.304291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4340cf5-d08d-47df-bdb6-63529d564a63 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.309227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.387121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.388833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.390027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.390980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.393560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.396709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.399753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 53/105] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:49.405339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.455061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.502666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815acda0-ec13-4774-b4bb-5598437a6766 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.526021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.585737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.588054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9e54d-15b5-4c85-91ef-651793c59b09 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.588886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.589367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.591183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.592126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:49.592653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.594558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.619043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 53/106] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:49.625754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.677627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.679455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.679742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.684595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.684595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.684595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.733549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 54/107] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:25:49 2022] POST /identity/v3/projects => generated 302 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:49.734621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.830979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082cd4c-b827-460e-bc14-0bcac7cd460f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.890978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.892903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.893427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.893900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] PATH_INFO: `/v3/users/444c58b9fd3b41a5ae80a4bdd6ac9b15` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.916093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.923638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] RBAC: Authorizing `identity:get_user(user_id=444c58b9fd3b41a5ae80a4bdd6ac9b15)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.925241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.926994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 54/108] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/users/444c58b9fd3b41a5ae80a4bdd6ac9b15 => generated 289 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:49.933790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.943126 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad1df9-1b41-4c56-838e-b219c35ec471 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.970825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.983608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.984726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.985193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.985644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.001017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:50.014422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:50.015331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:50.016773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:25:50.019671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 55/109] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:50.025820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.137340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3653c-b177-47f5-b374-2432ea318e23 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.159753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fad07-00de-4c81-9cff-90a971ab2f2b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.189059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85e31c-00dc-4bc8-b813-8dd5a327cb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.190196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85e31c-00dc-4bc8-b813-8dd5a327cb38 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.190864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85e31c-00dc-4bc8-b813-8dd5a327cb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.191281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85e31c-00dc-4bc8-b813-8dd5a327cb38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.200733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.201440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.201800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.202153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.202894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.203277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.204846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48c9dc-98e1-4a36-ba21-6376832ec2e7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.223810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 56/110] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.226034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a571794-ddd9-464d-99d8-e73b80ed5158 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.226401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a571794-ddd9-464d-99d8-e73b80ed5158 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.226859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a571794-ddd9-464d-99d8-e73b80ed5158 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.232645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 57/111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:50.233640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a571794-ddd9-464d-99d8-e73b80ed5158 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.276954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.284417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85e31c-00dc-4bc8-b813-8dd5a327cb38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.285916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85e31c-00dc-4bc8-b813-8dd5a327cb38 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.310652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.365623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a571794-ddd9-464d-99d8-e73b80ed5158 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.403160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8f6ce-f281-40ef-a427-a46f849803e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.403794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f6ce-f281-40ef-a427-a46f849803e9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.404116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f6ce-f281-40ef-a427-a46f849803e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.404394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f6ce-f281-40ef-a427-a46f849803e9 None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.405524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8f6ce-f281-40ef-a427-a46f849803e9 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.406963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8f6ce-f281-40ef-a427-a46f849803e9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.408582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 58/112] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.409691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 55/113] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:25:50.411366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfa9b2-ed3f-4bc5-bdd1-87b0e76eec9e None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.411840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfa9b2-ed3f-4bc5-bdd1-87b0e76eec9e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.412949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfa9b2-ed3f-4bc5-bdd1-87b0e76eec9e None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.414369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 56/114] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:50.418610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.418610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.418610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.418823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcc734-9391-4ee6-b7b6-992bb61c650f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.418823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcc734-9391-4ee6-b7b6-992bb61c650f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.418823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcc734-9391-4ee6-b7b6-992bb61c650f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.419689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 57/115] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:50.420385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcc734-9391-4ee6-b7b6-992bb61c650f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.438895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:50.440127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:50.519985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 59/116] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:50.520763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.527376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.538185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcc734-9391-4ee6-b7b6-992bb61c650f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.577753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.586962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c27081-5f67-4e91-8643-ecfa18ad4aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.588335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c27081-5f67-4e91-8643-ecfa18ad4aed None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.588832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c27081-5f67-4e91-8643-ecfa18ad4aed None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.589310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c27081-5f67-4e91-8643-ecfa18ad4aed None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.591411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c27081-5f67-4e91-8643-ecfa18ad4aed None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.593713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c27081-5f67-4e91-8643-ecfa18ad4aed None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.596394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 58/117] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.597887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.598641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.599373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.622300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:50.623972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:50.644049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecccb7-60c4-4757-b4be-218b9a332254 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.686765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21d3b6-a3fd-4476-b5b6-998525b4b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.687488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d3b6-a3fd-4476-b5b6-998525b4b15b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.687875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d3b6-a3fd-4476-b5b6-998525b4b15b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.688269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d3b6-a3fd-4476-b5b6-998525b4b15b None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.689585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21d3b6-a3fd-4476-b5b6-998525b4b15b None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.691986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21d3b6-a3fd-4476-b5b6-998525b4b15b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.699510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 60/118] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.699510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.699510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.699510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.712462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 59/119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:50.714111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.719879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:50.721221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:50.820840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.828274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573adb56-2f3b-4150-bff3-5d125c8c0d0b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.845433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 61/120] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:50.846495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.892168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.893934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.894617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.894902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.895186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/7db0d9ecb6fa4688a758312e6c7cd99a/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.898414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5711ce46bc304c3d9d171989b92e55bb, user_id=7db0d9ecb6fa4688a758312e6c7cd99a, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.900053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.920994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 60/121] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:50 2022] PUT /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/7db0d9ecb6fa4688a758312e6c7cd99a/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:50.922051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.028567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec68c9-8214-4847-91ef-34d100129eb8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.071427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2a624-db43-483b-ad4d-d43d13764ba3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.101660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.102471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.102680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.102961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.103716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] RBAC: Adding query filter params (role.id=7f0910382fdc47ad8396a6c442cda86d, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=444c58b9fd3b41a5ae80a4bdd6ac9b15) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.104049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.105671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.111271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.112187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.112637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.113059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.117442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 62/122] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/role_assignments?user.id=444c58b9fd3b41a5ae80a4bdd6ac9b15&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=7f0910382fdc47ad8396a6c442cda86d => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.126040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:51.133510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:51.134448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:51.135202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:51.137223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 61/123] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:51.143253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.198938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.225215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.319366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0543c-7d8b-4a56-b8f1-fc34b354d4be None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.327508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.356652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56931be3-3ee2-475f-84cd-b63fbf31a922 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.367805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.368719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.369350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.369689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.370757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.371077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.372914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.383396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 63/124] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.384195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.414147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.415350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.416200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.416950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.432818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:51.442865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:51.443937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:51.444877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:51.446418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 62/125] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:51.451654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.519192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200dde-6b10-47e9-be50-c9431fb4e5bb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.563626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.565551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.568716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.569286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.592528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:51.601441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:51.602311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:51.603069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:25:51.605883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 64/126] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:51.606881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.619525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.649535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabccb4-aea4-4145-bdf8-efdc1965bb08 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.686661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.687680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.688196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.688674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.690049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.691816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.693903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 63/127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.694739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.695476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.792350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc6f97-b1ab-4a3b-9330-bb83e9b562b9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.807646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875a373-0caa-457a-8339-d3af746e6cc2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.837856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.838491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.838778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.839056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.839736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.840065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.841466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.846525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.847387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.847912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.848428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.849354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.850446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.852124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.860143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 65/128] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.861054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.862170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 64/129] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.868903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.874441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.965859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.976559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ff617-3fae-434d-856f-157459db76a4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.017517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.018167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.018451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.018733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.020501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.022608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.024608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 66/130] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.025197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.097430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cec4c-42a7-434f-9197-34c433f457ee None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.144994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.145982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.147070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.147552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.148966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.150809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.152660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 65/131] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.153602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.184419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75892ecc-5f25-4da7-b4d6-14d93a39419f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.228056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.233629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.234287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.234576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.234858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.237188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.238651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.241568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 67/132] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.242445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.266857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188f33-d739-45cc-8e9a-b6d6e97be4c6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.303681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.303922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.306895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.306895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.306895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/bcc81a9e50c0416682e495a7f72fd265/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.308008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] RBAC: Authorizing `identity:create_grant(project_id=5711ce46bc304c3d9d171989b92e55bb, user_id=bcc81a9e50c0416682e495a7f72fd265, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.309590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.331038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 66/133] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:52 2022] PUT /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/bcc81a9e50c0416682e495a7f72fd265/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:52.331883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.437617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be43011-c842-4286-a5bd-042ebd1b25cf None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.508619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2445c-b253-40c9-84e8-830d6d4194ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.509637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2445c-b253-40c9-84e8-830d6d4194ae None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.510122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2445c-b253-40c9-84e8-830d6d4194ae None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.510598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2445c-b253-40c9-84e8-830d6d4194ae None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.512069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2445c-b253-40c9-84e8-830d6d4194ae None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.513739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2445c-b253-40c9-84e8-830d6d4194ae None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.517788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 68/134] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.518988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f298e-24d8-4c28-a972-e8cfa6b5584d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.519280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f298e-24d8-4c28-a972-e8cfa6b5584d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.519566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f298e-24d8-4c28-a972-e8cfa6b5584d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.523997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 69/135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:52.526444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f298e-24d8-4c28-a972-e8cfa6b5584d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.555848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acecd01-1107-46bf-915c-f1acbeba751e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.609130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.624501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e9f76-b6cb-4943-842c-2297b8ce34e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.625202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9f76-b6cb-4943-842c-2297b8ce34e6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.625509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9f76-b6cb-4943-842c-2297b8ce34e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.625793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e9f76-b6cb-4943-842c-2297b8ce34e6 None admin] PATH_INFO: `/v3/projects/168ba65d920d4d1da42783d12e1e142b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.632733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e9f76-b6cb-4943-842c-2297b8ce34e6 None admin] RBAC: Authorizing `identity:get_project(project_id=168ba65d920d4d1da42783d12e1e142b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.634339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e9f76-b6cb-4943-842c-2297b8ce34e6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.636341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 67/136] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/projects/168ba65d920d4d1da42783d12e1e142b => generated 312 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.637350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.637685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.637911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.663226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.672358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f298e-24d8-4c28-a972-e8cfa6b5584d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.672779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:52.674111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:52.713078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.714280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.714714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.715232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/7db0d9ecb6fa4688a758312e6c7cd99a/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.718170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5711ce46bc304c3d9d171989b92e55bb, user_id=7db0d9ecb6fa4688a758312e6c7cd99a, role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.719798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.734817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 70/137] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:52 2022] PUT /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/7db0d9ecb6fa4688a758312e6c7cd99a/roles/87236176cec04669b7f8d8199ce12228 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:52.736142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.873718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ac213-880b-49a0-8607-9610180ed4e0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.882389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 68/138] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:52.914677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.915715 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.916205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.916748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] PATH_INFO: `/v3/projects/168ba65d920d4d1da42783d12e1e142b/users/7db0d9ecb6fa4688a758312e6c7cd99a/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.919252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] RBAC: Authorizing `identity:create_grant(project_id=168ba65d920d4d1da42783d12e1e142b, user_id=7db0d9ecb6fa4688a758312e6c7cd99a, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.920927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.939513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 71/139] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:52 2022] PUT /identity/v3/projects/168ba65d920d4d1da42783d12e1e142b/users/7db0d9ecb6fa4688a758312e6c7cd99a/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:53.328718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:53.448268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314a003-2c29-4a84-bdfe-eac570b8cc77 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:53.484521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:53.490883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5b49e-e449-4193-8805-5e5f3d9a66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:53.496682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5b49e-e449-4193-8805-5e5f3d9a66b2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.496980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5b49e-e449-4193-8805-5e5f3d9a66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.497262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5b49e-e449-4193-8805-5e5f3d9a66b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.498450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5b49e-e449-4193-8805-5e5f3d9a66b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:53.499915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5b49e-e449-4193-8805-5e5f3d9a66b2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:53.501833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 69/140] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:53.507554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:53.531782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:53.616848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e434-e332-4262-be6b-faba6bf3cf81 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.617112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e434-e332-4262-be6b-faba6bf3cf81 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.617407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e434-e332-4262-be6b-faba6bf3cf81 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.618886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 70/141] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:53.624513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99642922-9850-4513-8f2a-399a4ceacd1f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.625071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99642922-9850-4513-8f2a-399a4ceacd1f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.625071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99642922-9850-4513-8f2a-399a4ceacd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.637751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d6590-2c9f-4afd-bbce-bb379d95c82c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:53.649201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-99642922-9850-4513-8f2a-399a4ceacd1f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:53.650836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-99642922-9850-4513-8f2a-399a4ceacd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:53.675222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8637ed5-c57d-4d73-85cd-b3ba75afae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:53.676559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8637ed5-c57d-4d73-85cd-b3ba75afae6e None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.676914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8637ed5-c57d-4d73-85cd-b3ba75afae6e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.677286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8637ed5-c57d-4d73-85cd-b3ba75afae6e None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.678485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8637ed5-c57d-4d73-85cd-b3ba75afae6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:53.680163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8637ed5-c57d-4d73-85cd-b3ba75afae6e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:53.717742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 72/142] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:53 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:25:53.746781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 71/143] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:54.052448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169a0a7-8b7a-4a33-8c29-f8990e222289 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.053413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169a0a7-8b7a-4a33-8c29-f8990e222289 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.053943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169a0a7-8b7a-4a33-8c29-f8990e222289 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.054959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 73/144] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:54.069165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.069355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.069566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.093418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:54.094616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:54.181299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169a0a7-8b7a-4a33-8c29-f8990e222289 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.184934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 72/145] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:54.288666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.326378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169a0a7-8b7a-4a33-8c29-f8990e222289 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.369210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.369860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.370170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.370471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.384323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:54.389133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:54.389967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:25:54.390738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 74/146] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:54.399186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.472000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.517630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102e62-3598-4bd7-a6ab-4f358932157a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.559483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.560491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.560984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.561461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.562339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:54.562846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.564472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.574806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 73/147] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:54.580473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.591903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.762463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8787-642c-45bf-9e56-b0b84ca5e121 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.800315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.821676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.822655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.823152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.823639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.838310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566211a-9eb3-4c56-a650-926924dfb0e0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.842063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:54.848488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:54.849628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:54.850776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:54.857170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 74/148] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:54.864084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.914042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.932751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.933691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.934136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.934572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] PATH_INFO: `/v3/users/444c58b9fd3b41a5ae80a4bdd6ac9b15` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.936693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] RBAC: Authorizing `identity:get_user(user_id=444c58b9fd3b41a5ae80a4bdd6ac9b15)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.938336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.943719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 75/149] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/users/444c58b9fd3b41a5ae80a4bdd6ac9b15 => generated 289 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:54.949717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.033777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba2b01-93e2-4c7e-9ed2-0aa858bb3ff4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.092824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.093386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.093669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.093947 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.094637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:55.094942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.096384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.105986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 75/150] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:55.111629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.176294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.186768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c04983-78d0-4452-adac-ff09ddd820b7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.253870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e878e3-b9f0-4c9c-8cf1-ff42271118e0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.288691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.289353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.289642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.289920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.291365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.293191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.295061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 76/151] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:55.300814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.304601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.305257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.305541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.305819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.307843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.309367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.315690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 76/152] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:55.322953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.454141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.516374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465847e9-90eb-47a8-8379-cdc4f7f47b5d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.536232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.539913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c974906-483a-4c60-b866-cf8cd9ef554b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.605330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.607041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.608392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.609283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.638061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d25aa4-3e21-4cf7-9f0e-e54482d2436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.639071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d25aa4-3e21-4cf7-9f0e-e54482d2436d None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.639541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d25aa4-3e21-4cf7-9f0e-e54482d2436d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.639992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d25aa4-3e21-4cf7-9f0e-e54482d2436d None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/444c58b9fd3b41a5ae80a4bdd6ac9b15/roles/7f0910382fdc47ad8396a6c442cda86d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.642961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d25aa4-3e21-4cf7-9f0e-e54482d2436d None admin] RBAC: Authorizing `identity:create_grant(project_id=5711ce46bc304c3d9d171989b92e55bb, user_id=444c58b9fd3b41a5ae80a4bdd6ac9b15, role_id=7f0910382fdc47ad8396a6c442cda86d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.645763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d25aa4-3e21-4cf7-9f0e-e54482d2436d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.670122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 77/153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:55 2022] PUT /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb/users/444c58b9fd3b41a5ae80a4bdd6ac9b15/roles/7f0910382fdc47ad8396a6c442cda86d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:55.671850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48025819-b1ba-4e2e-b5b5-0af22b0fac2c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.673587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48025819-b1ba-4e2e-b5b5-0af22b0fac2c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.674044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48025819-b1ba-4e2e-b5b5-0af22b0fac2c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.676540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 78/154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:55.679339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee1baa-3621-4b11-acac-c0fcf512e293 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.682933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee1baa-3621-4b11-acac-c0fcf512e293 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.683214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee1baa-3621-4b11-acac-c0fcf512e293 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.684697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 79/155] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:55.685860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4366a-dc98-494f-a5fb-35fbca3c0950 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.686144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4366a-dc98-494f-a5fb-35fbca3c0950 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.686421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4366a-dc98-494f-a5fb-35fbca3c0950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.717252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-00b4366a-dc98-494f-a5fb-35fbca3c0950 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:55.718522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-00b4366a-dc98-494f-a5fb-35fbca3c0950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:55.751709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.753610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.857288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 80/156] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:55.859317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8fcf4-2236-4d52-b631-4c02b0cd34c4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.859674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8fcf4-2236-4d52-b631-4c02b0cd34c4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.860132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8fcf4-2236-4d52-b631-4c02b0cd34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.889393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-22e8fcf4-2236-4d52-b631-4c02b0cd34c4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:55.890798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-22e8fcf4-2236-4d52-b631-4c02b0cd34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:55.978341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.992751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 77/157] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:55.998964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.014534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 81/158] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:56.024294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.109901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8108d-4586-48b9-be49-b2d74dd4333a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.110523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8108d-4586-48b9-be49-b2d74dd4333a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.111039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8108d-4586-48b9-be49-b2d74dd4333a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.114681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 82/159] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:56.120775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.121331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.121966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.140283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:56.141923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:56.214720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8cf76-8400-4ffa-bf42-2d4154065f25 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.241900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 83/160] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:56.283990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.284635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.284913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.285184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.288283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.290201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.292424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 78/161] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:56.299991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.436452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9b0dc-37bc-4d90-ad4d-fe2217a5360a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.454405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.483473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.484121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.484403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.484677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.485410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=7db0d9ecb6fa4688a758312e6c7cd99a) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:56.485718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.487157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.493807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 84/162] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/role_assignments?user.id=7db0d9ecb6fa4688a758312e6c7cd99a&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:56.567639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.633029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.704320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.745017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd1677-c83f-43ff-8d94-40cde2a743b7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.812992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.813760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.814034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.814297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.833253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:56.844058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43496b8d-9485-46cd-b6e7-15c0197809ac None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:56.845887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:56.846757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 79/163] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:56.847937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.901776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.902442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.902707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.903156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.917043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:56.925711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:56.926548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:25:56.927357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 85/164] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:56.928842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.021242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.058874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e8abd-7589-4615-912f-9cab9ab51cd2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.112258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c746ac-0404-4a26-a575-8a7fd4e729bd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.133823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.134582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.134863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.135138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.151768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.168783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:57.172663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.173296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.173629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:57.173850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:57.175919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:57.178736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:57.184317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.184317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.184317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.184762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 80/165] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:57.184762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.185039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.186334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 86/166] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.187280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.362583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20006cb-a81c-45cd-9188-64e5192dcaf4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.364182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.386433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58aa5d-21c6-4138-934f-9e094e239a58 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.426093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.426779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.427066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.427346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.428257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.428734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.430153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.452096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.452915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 87/167] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.453108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.453108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.453108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.454199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.454366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.454525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.456096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.470185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 81/168] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.470185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.576400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.666789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475f72-d57f-4275-9f5c-da8131cb0bd1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.695119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0d30-6035-433b-bfa3-d4707665a838 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.719589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.720251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.720712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.721150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.722434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.724059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.726375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 88/169] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.731798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.769815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.770580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.770932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.771843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] PATH_INFO: `/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.773117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] RBAC: Authorizing `identity:get_user(user_id=7db0d9ecb6fa4688a758312e6c7cd99a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.774660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.784253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.784966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 82/170] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users/7db0d9ecb6fa4688a758312e6c7cd99a => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.786038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.898371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9df4e7-9b32-4463-be3d-53cdf2a897d8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.960215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ed9e8-49fe-4d84-a5e5-2b41fb03fb4d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.973600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.974254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.975063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.975504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] PATH_INFO: `/v3/users/bcc81a9e50c0416682e495a7f72fd265` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.976934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] RBAC: Authorizing `identity:get_user(user_id=bcc81a9e50c0416682e495a7f72fd265)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.978526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.988400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 89/171] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users/bcc81a9e50c0416682e495a7f72fd265 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.989563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.001400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.013676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.014441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.014826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.015201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.153909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.155407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.156293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.159905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96044f-9e2e-4e8e-a838-ea2de0829d9a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.228905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb28bca-3ffe-4800-ade5-4e5e1e932790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.229454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb28bca-3ffe-4800-ade5-4e5e1e932790 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.229740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb28bca-3ffe-4800-ade5-4e5e1e932790 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.230021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb28bca-3ffe-4800-ade5-4e5e1e932790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.324568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 83/172] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:58.328288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.369463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.383992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb28bca-3ffe-4800-ade5-4e5e1e932790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.383992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb28bca-3ffe-4800-ade5-4e5e1e932790 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.495453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c612c-10c5-41a8-bb5f-2d3a88f10ca8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.504256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 90/173] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:58.504779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.509457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd196a-1953-49d2-a97f-18549da782e6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.509981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd196a-1953-49d2-a97f-18549da782e6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.510426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd196a-1953-49d2-a97f-18549da782e6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.514035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 91/174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:58.514941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd196a-1953-49d2-a97f-18549da782e6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.547402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76491274-81e2-40a5-977d-14a26dbd056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.548049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76491274-81e2-40a5-977d-14a26dbd056d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.548332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76491274-81e2-40a5-977d-14a26dbd056d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.548607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76491274-81e2-40a5-977d-14a26dbd056d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.636480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76491274-81e2-40a5-977d-14a26dbd056d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.637946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76491274-81e2-40a5-977d-14a26dbd056d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.656261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd196a-1953-49d2-a97f-18549da782e6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.666643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.695658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbb05e-932b-45dd-8934-0f3124af994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.696326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbb05e-932b-45dd-8934-0f3124af994a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.696607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbb05e-932b-45dd-8934-0f3124af994a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.696882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbb05e-932b-45dd-8934-0f3124af994a None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.698004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbb05e-932b-45dd-8934-0f3124af994a None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.699618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbb05e-932b-45dd-8934-0f3124af994a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.701139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 92/175] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:58.702522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935959a-67d5-4d42-b112-7bd4b57f331d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.702805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935959a-67d5-4d42-b112-7bd4b57f331d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.703081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935959a-67d5-4d42-b112-7bd4b57f331d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.704389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 93/176] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:58.705328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935959a-67d5-4d42-b112-7bd4b57f331d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.764675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 84/177] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:58.764675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524620d-a2fa-4248-b208-edb73536415f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.764675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524620d-a2fa-4248-b208-edb73536415f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.764675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524620d-a2fa-4248-b208-edb73536415f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.780144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a524620d-a2fa-4248-b208-edb73536415f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:58.781414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a524620d-a2fa-4248-b208-edb73536415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:58.786398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.825091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935959a-67d5-4d42-b112-7bd4b57f331d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.856086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 85/178] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:58.857409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89086d-e3d6-4d9a-9a93-92ebb8b757e7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.857695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89086d-e3d6-4d9a-9a93-92ebb8b757e7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.857994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89086d-e3d6-4d9a-9a93-92ebb8b757e7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.859425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 86/179] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:58.860041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89086d-e3d6-4d9a-9a93-92ebb8b757e7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.876544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cc7b7-8fb3-49b8-86ad-7f6d8e67882d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.877540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cc7b7-8fb3-49b8-86ad-7f6d8e67882d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.877998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cc7b7-8fb3-49b8-86ad-7f6d8e67882d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.878591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cc7b7-8fb3-49b8-86ad-7f6d8e67882d None admin] PATH_INFO: `/v3/projects/168ba65d920d4d1da42783d12e1e142b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.880126 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cc7b7-8fb3-49b8-86ad-7f6d8e67882d None admin] RBAC: Authorizing `identity:get_project(project_id=168ba65d920d4d1da42783d12e1e142b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.882630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cc7b7-8fb3-49b8-86ad-7f6d8e67882d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.884639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 94/180] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/projects/168ba65d920d4d1da42783d12e1e142b => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:58.886193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.886753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.887227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.906806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:58.908388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:58.925776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.972715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89086d-e3d6-4d9a-9a93-92ebb8b757e7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.985217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 95/181] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:58.986657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.012198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.012813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.013088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.013360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.014093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] RBAC: Adding query filter params (role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=7db0d9ecb6fa4688a758312e6c7cd99a) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.014447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.016534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a03d-ef42-427f-bd40-79c1e0a14e2e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.023400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 87/182] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/role_assignments?user.id=7db0d9ecb6fa4688a758312e6c7cd99a&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=87236176cec04669b7f8d8199ce12228 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.024966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.025281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.025525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.045694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:59.047113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:59.070195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.109950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0879c8-2eb1-4a9a-b46f-829f1092c85e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.127042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 88/183] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:59.131986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.172664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8cb7d-050e-4a5d-83f2-384009d4b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.173719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8cb7d-050e-4a5d-83f2-384009d4b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.174212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8cb7d-050e-4a5d-83f2-384009d4b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.174688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8cb7d-050e-4a5d-83f2-384009d4b7c3 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.176081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8cb7d-050e-4a5d-83f2-384009d4b7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.178426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8cb7d-050e-4a5d-83f2-384009d4b7c3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.180617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 96/184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.181960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef566-6083-4f5d-a6bd-117fbb9f4e41 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.182443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef566-6083-4f5d-a6bd-117fbb9f4e41 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.182914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef566-6083-4f5d-a6bd-117fbb9f4e41 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.184627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 97/185] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:59.190449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.193235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef566-6083-4f5d-a6bd-117fbb9f4e41 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.248137 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62a350-d140-4b6a-8bad-0da0db1165aa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.294036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.295511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.295800 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.296931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.297734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=168ba65d920d4d1da42783d12e1e142b, user.id=7db0d9ecb6fa4688a758312e6c7cd99a) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.298041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.299710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b5ab-e6cb-447f-83d7-fdede0fa8398 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.309404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 89/186] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/role_assignments?user.id=7db0d9ecb6fa4688a758312e6c7cd99a&scope.project.id=168ba65d920d4d1da42783d12e1e142b&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.310551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.315973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.315973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.332784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:59.334314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:59.356532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef566-6083-4f5d-a6bd-117fbb9f4e41 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.423319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.424671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.425185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.425949 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.428308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=bcc81a9e50c0416682e495a7f72fd265) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.428869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.431369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.434084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.437490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 90/187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:59.438036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.443026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 98/188] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/role_assignments?user.id=bcc81a9e50c0416682e495a7f72fd265&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.444375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.516675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.589740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b241a-3ada-45ee-8c4d-077c952723ab None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.591299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e77c-7f46-4c4b-a64a-66b394c151f7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.643277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.643938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.644256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.644608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.647290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.648598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.649128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.649604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.651583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.653658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.655578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 99/189] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.661372 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:59.662197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:59.668512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:59.669419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:25:59.670301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 91/190] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:59.676672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.746328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.786426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b325b26-82d0-41bf-8b60-660ea4104634 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.800900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dc8f-16e2-44b0-b1d5-ea7a3e181cad None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.809428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.832401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.832401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.832401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.832401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.832802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.834858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.849159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.850634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.851207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.851386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.852205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.852522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.854128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.862507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 100/191] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:59 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:25:59.864155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.865756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 92/192] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:59.871738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.006574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01232-89ed-443a-a2f7-d481781a1ccf None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.009891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591854-097f-4108-ab6e-f815ee89cd5e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.031702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.051640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.052338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.053132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.053467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.057020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.057656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.057935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.058204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] PATH_INFO: `/v3/users/11673db0a9f248858411797c26633075` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.069317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:00.074840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=11673db0a9f248858411797c26633075)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:00.075037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:00.075912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:00.076795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:00.077402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.077478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 101/193] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:00.077478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.082866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 93/194] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/users/11673db0a9f248858411797c26633075 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:00.083931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.119133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.207638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02b1c-2cfd-41e8-bce6-5904bf7b246a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.219401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b021e0-9242-42fa-880c-076555461cc3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.248817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.250320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.250884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.251337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.261922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.262730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.263086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.263434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.264231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:00.264604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.266057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.266735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:00.274054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:00.274930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:00.275725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:00.276356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 94/195] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.277944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 102/196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:00.280084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.281929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.348350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.393932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.400678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79565c78-8f0d-4318-a99e-57108c135ea7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.414765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfd4e0-4955-4404-90d1-8b3ce3b1b4e1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.442086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.443326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.443752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.444256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] PATH_INFO: `/v3/users/11673db0a9f248858411797c26633075` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.446435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] RBAC: Authorizing `identity:get_user(user_id=11673db0a9f248858411797c26633075)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.448767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.451538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 103/197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/users/11673db0a9f248858411797c26633075 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.452537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.459746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.460215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.460598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.460972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.544677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.546443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.581384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c9e97-ed45-4eeb-8463-7d43e0b4ade1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.596382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.619615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.620547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.620927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.621464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.622185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:00.622634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.624331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.634235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 104/198] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.635320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.661475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.678236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 95/199] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:00.679381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.763197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baf9b8-ba61-49cb-be6f-c96ffa56ea1b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.786495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eff0e-3abc-424e-a769-a68f65380557 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.810015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.810857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.811545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.812004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.825316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.825795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.826175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.826550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] PATH_INFO: `/v3/users/444c58b9fd3b41a5ae80a4bdd6ac9b15` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.827824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] RBAC: Authorizing `identity:get_user(user_id=444c58b9fd3b41a5ae80a4bdd6ac9b15)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.829547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.831249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 96/200] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/users/444c58b9fd3b41a5ae80a4bdd6ac9b15 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.832127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.865795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.892380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.901579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.903237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.939440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f038d46-5d32-4a97-acdb-b6bf4a5ff731 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.992414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.992414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.992414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.992414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.992809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.992809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.993699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 97/201] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.994471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.034167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 105/202] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:01.036422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.106183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a1c9-0c18-46bd-ab44-78a61aea2f52 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.119165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.145859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.154618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa14262-336d-4006-a3b9-7c45e45a2014 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.161611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.163194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.163682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.164195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.191226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.191776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.192107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.192384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.193741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=11673db0a9f248858411797c26633075) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.194055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.195578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.202147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 106/203] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/role_assignments?user.id=11673db0a9f248858411797c26633075&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.203086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.277084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.278577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.398937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2af0de-15a2-4824-b717-2be6116f55f1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.461420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 98/204] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:01.466571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.467235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.467518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.467794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.468901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.469081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.470491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.472487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 107/205] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.484996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.495213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.498992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.611644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991f892-2eb8-4370-8dd5-60c657ddf2fd None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.647492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.648158 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.648456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.648715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.649872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.651317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.656768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1cad-d245-4654-b915-bf4ab36aff47 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.657840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 99/206] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.659025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.725385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.725833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.726124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.726505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.727358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] RBAC: Adding query filter params (role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=11673db0a9f248858411797c26633075) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.727676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.729173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee94ce-156f-4386-91b2-214db57fb79a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.747155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 108/207] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/role_assignments?user.id=11673db0a9f248858411797c26633075&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=87236176cec04669b7f8d8199ce12228 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.764339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.776392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafaaf8-7694-47ee-b2ee-bba3e1039545 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.832150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.832770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.833051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.833326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.834145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] RBAC: Adding query filter params (role.id=7f0910382fdc47ad8396a6c442cda86d, scope.project.id=5711ce46bc304c3d9d171989b92e55bb, user.id=444c58b9fd3b41a5ae80a4bdd6ac9b15) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.834456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.836065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752355f6-5219-4814-ba0f-392b74014e52 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.848104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 100/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/role_assignments?user.id=444c58b9fd3b41a5ae80a4bdd6ac9b15&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=7f0910382fdc47ad8396a6c442cda86d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:02.202295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a196177-2bd5-426e-9dab-a9fe4a67e23a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.202798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a196177-2bd5-426e-9dab-a9fe4a67e23a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.202872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a196177-2bd5-426e-9dab-a9fe4a67e23a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.204212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 109/209] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:02.209752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.210037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.210309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.231386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a75d8c-e524-4f94-a3ad-290a9733f363 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.231687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a75d8c-e524-4f94-a3ad-290a9733f363 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.231964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a75d8c-e524-4f94-a3ad-290a9733f363 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.233427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 110/210] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:02.233670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:02.234946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:02.239277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.239802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.240951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.263724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:02.265546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:02.340123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 101/211] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:02.360422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.368706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 111/212] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:02.614483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.657723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.705819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.725319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a2233-7c3e-4e83-b4a8-b2e738239db7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.766603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.767684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.768209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.768680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.780062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdca475-5012-48aa-bb8d-978c7d7d5851 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.780207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:02.785201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:02.786050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:02.787526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:02.795047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 102/213] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:02.796180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.820292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.821210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.821496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.821768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.823044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.825145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.826914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 112/214] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:02.832301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.914812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.923690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127ee25-0d05-496e-9d0c-7fbff27a54e1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.957954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434eaba-b411-42e8-81de-508e0c85e8de None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.969017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.969627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.969904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.970197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.970848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:02.971150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.972567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.986512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 103/215] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:02.987541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.007312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.017325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.018082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.018498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.018782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.019444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.021611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.057485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 113/216] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:02 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:26:03.132957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ed33c-7593-412f-9354-bc3458dfc453 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.173192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec394b8-9d5d-424d-b493-28102b764587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.174101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec394b8-9d5d-424d-b493-28102b764587 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.174659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec394b8-9d5d-424d-b493-28102b764587 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.175067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec394b8-9d5d-424d-b493-28102b764587 None admin] PATH_INFO: `/v3/users/b26ec8597f7f47db996ed183a77d2842` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.192802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec394b8-9d5d-424d-b493-28102b764587 None admin] RBAC: Authorizing `identity:get_user(user_id=b26ec8597f7f47db996ed183a77d2842)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.195257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec394b8-9d5d-424d-b493-28102b764587 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.197493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 104/217] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/users/b26ec8597f7f47db996ed183a77d2842 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:03.205309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.331932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2a869-1d49-4b35-b996-e4c9584e6d05 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.357393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.372461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.373323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.373824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.374197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.393847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6ccf7-9930-4fbd-9e7c-8fad5fd6f6fb None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.394797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6ccf7-9930-4fbd-9e7c-8fad5fd6f6fb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.395286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6ccf7-9930-4fbd-9e7c-8fad5fd6f6fb None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.397009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 105/218] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:03.403297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.403707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.404115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.426878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:03.428679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:03.462943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.464474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.528820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.548285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 106/219] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:03.649451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 114/220] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:03.660137 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.827039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a41a8-2431-4452-8ba3-6f9560183fa3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.877138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.877787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.878068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.878341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.879471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.881661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.883757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 107/221] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:03.889679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.910461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.937142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.984140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.029633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba548c0-6630-4c76-ab9f-c164be233ee9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.049692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f066ab-a8ac-41d0-aea3-88551e2714bf None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.070789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041b2d6-93f2-495a-a595-8653e938aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.071715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041b2d6-93f2-495a-a595-8653e938aa2a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.072208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041b2d6-93f2-495a-a595-8653e938aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.072653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041b2d6-93f2-495a-a595-8653e938aa2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.083591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041b2d6-93f2-495a-a595-8653e938aa2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8041b2d6-93f2-495a-a595-8653e938aa2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:04.088651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:04.089530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:04.090344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:04.092672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 108/222] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:04.094150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31691-6e78-4240-bd10-9e7883c933f3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.094634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31691-6e78-4240-bd10-9e7883c933f3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.095124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31691-6e78-4240-bd10-9e7883c933f3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.096618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 109/223] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:04.101274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.101274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.101274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.101274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.101650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=b26ec8597f7f47db996ed183a77d2842) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:04.101650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.101961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c37e-f544-4431-9802-6959e07e0a1e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.102925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31691-6e78-4240-bd10-9e7883c933f3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.109415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 115/224] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/role_assignments?user.id=b26ec8597f7f47db996ed183a77d2842&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:04.110921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.111480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.112002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.158610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:04.159854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:04.246518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 116/225] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:04.304127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31691-6e78-4240-bd10-9e7883c933f3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.328138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.342281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.342582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.342865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.343138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.343820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:04.344724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.345552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.356141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 110/226] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:04.361938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.391348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.504304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef57f7-7a0d-4f26-8572-4e0329fbde39 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.547031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.547687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.547984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.548265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] PATH_INFO: `/v3/users/11673db0a9f248858411797c26633075` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.549428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=11673db0a9f248858411797c26633075)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.550879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.553270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 117/227] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/users/11673db0a9f248858411797c26633075 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:04.559069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.594074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.714933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.735043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31af7c-9afd-44f7-ae52-7111ae74d5e9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.764579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e7979-abc8-4c16-ba99-e47ec0725dd0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.811884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.812577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.813308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.813597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.826320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:04.831135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:04.832038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:04.832836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 118/228] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:04.841039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.846574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.847172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.847451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.847730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.849847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.851468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.857959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 111/229] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:04.864978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.968221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.044095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3b7da-3a6a-48d8-8e37-63ea1f401947 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.056903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcb0ee-81d1-4d34-a4a3-63ea591b2e93 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.107965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.109978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.110264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.110542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.111220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:05.111523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.112930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.121771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.122446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.122720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.123052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/11673db0a9f248858411797c26633075/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.125037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6897c5d1cf24f76b49b423cc3b01774, user_id=11673db0a9f248858411797c26633075, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.126539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.130594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 119/230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.139285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.152651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 112/231] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:04 2022] PUT /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/11673db0a9f248858411797c26633075/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:26:05.154510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.296805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823bcd-1596-4250-9ea1-62d7432538ac None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.344827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.345901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.346384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.346865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] PATH_INFO: `/v3/users/11673db0a9f248858411797c26633075` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.348280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] RBAC: Authorizing `identity:get_user(user_id=11673db0a9f248858411797c26633075)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.349978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.352134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 120/232] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/users/11673db0a9f248858411797c26633075 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.358863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.475266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0505db-5593-47a9-99f6-5609c224fe64 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.493073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.525414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1170479-0b98-4d00-afe1-c41257f6a7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.526054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1170479-0b98-4d00-afe1-c41257f6a7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.526334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1170479-0b98-4d00-afe1-c41257f6a7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.526608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1170479-0b98-4d00-afe1-c41257f6a7b4 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.532655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1170479-0b98-4d00-afe1-c41257f6a7b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.534116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1170479-0b98-4d00-afe1-c41257f6a7b4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.536144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 113/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.542381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.664773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.704587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23087c-cf56-4b63-b6ee-a53012047c22 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.753195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385d8d2-f7bb-45e0-a28a-0e5e3600c942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.753873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385d8d2-f7bb-45e0-a28a-0e5e3600c942 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.754221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385d8d2-f7bb-45e0-a28a-0e5e3600c942 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.754499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385d8d2-f7bb-45e0-a28a-0e5e3600c942 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/11673db0a9f248858411797c26633075/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.762035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385d8d2-f7bb-45e0-a28a-0e5e3600c942 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6897c5d1cf24f76b49b423cc3b01774, user_id=11673db0a9f248858411797c26633075, role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.763511 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385d8d2-f7bb-45e0-a28a-0e5e3600c942 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.784078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 121/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:05 2022] PUT /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/11673db0a9f248858411797c26633075/roles/87236176cec04669b7f8d8199ce12228 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:05.812546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84379-bc80-4acb-9471-aa4ec581f1e9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.813062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84379-bc80-4acb-9471-aa4ec581f1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.813134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84379-bc80-4acb-9471-aa4ec581f1e9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.814699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 114/235] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:05.824953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.825265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.825516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.845569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:05.846817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:05.870581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876449e9-9fb7-4d8e-adf5-11f82c652dc0 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.871199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876449e9-9fb7-4d8e-adf5-11f82c652dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.871670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876449e9-9fb7-4d8e-adf5-11f82c652dc0 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.873534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 115/236] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:05.879472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.880154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.881344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.901745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:05.903425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:05.924496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 122/237] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:05.983455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 116/238] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:06.235424 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.316732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.366786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c767ac3-39a0-4ee9-a5c4-e1d71cda21e5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.413822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.414984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.415270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.415545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.416709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.418490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.420399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 123/239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:06.425770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.435938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.550759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816dd8-551a-49b4-b0ca-7ea593b37cc5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.584755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.590843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed22386-1062-40e7-a341-8f831c2960c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.591490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed22386-1062-40e7-a341-8f831c2960c9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.591771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed22386-1062-40e7-a341-8f831c2960c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.592186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed22386-1062-40e7-a341-8f831c2960c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.602875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db804060-5ea6-4be8-acf9-07a8cca2795a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.604413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed22386-1062-40e7-a341-8f831c2960c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6ed22386-1062-40e7-a341-8f831c2960c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:06.609326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:06.610177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:06.610985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 124/240] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:06.613861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a1410-2b87-4e3a-8527-9260abc6a5a0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.613861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a1410-2b87-4e3a-8527-9260abc6a5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.613861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a1410-2b87-4e3a-8527-9260abc6a5a0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.616223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 125/241] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:06.621369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a1410-2b87-4e3a-8527-9260abc6a5a0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.622891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.645745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0d71c-5733-4a07-948b-690f784180a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.646893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d71c-5733-4a07-948b-690f784180a0 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.647334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d71c-5733-4a07-948b-690f784180a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.647618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d71c-5733-4a07-948b-690f784180a0 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.648310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0d71c-5733-4a07-948b-690f784180a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.649766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0d71c-5733-4a07-948b-690f784180a0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.679974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 117/242] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:06 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:06.679974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.679974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.679974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.697996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:06.699245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:06.779909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 118/243] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:06.797929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a1410-2b87-4e3a-8527-9260abc6a5a0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.845874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.846789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.847315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.847769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.848674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:06.849192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.851387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.861677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 126/244] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:06.873138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.994912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.004030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4e8c9-10ad-40b5-9ffb-24a084ab7c99 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.046206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.046851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.047136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.047416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] PATH_INFO: `/v3/users/dafa9f6799db46b7a61b499b6df9d5c8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.060926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] RBAC: Authorizing `identity:get_user(user_id=dafa9f6799db46b7a61b499b6df9d5c8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.062399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.064284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 119/245] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/users/dafa9f6799db46b7a61b499b6df9d5c8 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:07.070556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.121198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.138016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.190124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6a0d-e8c6-4e13-b365-0335ed7191c6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.230459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.231100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.231757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.231864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.241937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6ff61-dbd2-48b7-86cd-efa364715d15 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.280422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.281501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.281977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.282450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.294853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:07.300893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:07.301770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:07.303378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:07.310836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.310983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.311216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 120/246] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:07.312679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.337782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.377935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.428908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 127/247] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:07.434873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.438722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d53ae-e7e9-43b5-b923-b08bbb55e49e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.478040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.479023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.479842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.480351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.481572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:07.482350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.484142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.500835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 121/248] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:07.506175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.568771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c615ef-6b61-44b2-aca1-b01c69b49aee None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.614657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.620217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.621678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.622036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.622387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.623902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.626248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.632038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 128/249] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:07.640671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.650342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381bbce-9c7d-418d-aa59-40f8d2d159e0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.678686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.694346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737fe37-fe3c-4f6a-8459-4c9a8f6ffc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.694996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737fe37-fe3c-4f6a-8459-4c9a8f6ffc05 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.695270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737fe37-fe3c-4f6a-8459-4c9a8f6ffc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.695539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737fe37-fe3c-4f6a-8459-4c9a8f6ffc05 None admin] PATH_INFO: `/v3/users/b26ec8597f7f47db996ed183a77d2842` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.697504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737fe37-fe3c-4f6a-8459-4c9a8f6ffc05 None admin] RBAC: Authorizing `identity:get_user(user_id=b26ec8597f7f47db996ed183a77d2842)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.698169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737fe37-fe3c-4f6a-8459-4c9a8f6ffc05 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.699921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 122/250] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/users/b26ec8597f7f47db996ed183a77d2842 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:07.700919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccbf89-6a99-4ded-894c-e4eda0f0196a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.701198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccbf89-6a99-4ded-894c-e4eda0f0196a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.701468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccbf89-6a99-4ded-894c-e4eda0f0196a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.702661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 123/251] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:07.708079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.708343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.708655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.728947 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:07.730716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:07.783192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0889ca5-793c-4430-a8e7-026345d6922f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.825679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 124/252] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:07.826777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.827190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.827398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.827669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.827936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.828829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] RBAC: Adding query filter params (role.id=7f0910382fdc47ad8396a6c442cda86d, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=dafa9f6799db46b7a61b499b6df9d5c8) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:07.829140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.830759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.837873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 129/253] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/role_assignments?user.id=dafa9f6799db46b7a61b499b6df9d5c8&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=7f0910382fdc47ad8396a6c442cda86d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:07.924710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.953622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e2015-e1db-4117-b680-45a3cf95d708 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.993216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.994192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.994697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.995151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.996547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.998186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.000181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 125/254] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:08.007358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.135767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.153878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d75087-34d7-4b52-903a-8f9dcb75422d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.202136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cec953-aaa8-42cd-9126-781abbdf8e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.202743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec953-aaa8-42cd-9126-781abbdf8e58 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.203010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec953-aaa8-42cd-9126-781abbdf8e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.203289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec953-aaa8-42cd-9126-781abbdf8e58 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/b26ec8597f7f47db996ed183a77d2842/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.211459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cec953-aaa8-42cd-9126-781abbdf8e58 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6897c5d1cf24f76b49b423cc3b01774, user_id=b26ec8597f7f47db996ed183a77d2842, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:08.212937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cec953-aaa8-42cd-9126-781abbdf8e58 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.247798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 130/255] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:08 2022] PUT /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/b26ec8597f7f47db996ed183a77d2842/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:08.355413 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.356363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34906ae2-6968-4193-853e-10301381a91f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.356652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34906ae2-6968-4193-853e-10301381a91f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.356933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34906ae2-6968-4193-853e-10301381a91f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.358419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 131/256] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:08.363921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.364228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.364505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.397182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:08.398589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:08.519448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb6eb5-a2b3-434d-b097-122636ed251c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.570631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.571279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.571556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.571829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.585588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 132/257] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:08.589543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:08.594334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:08.596398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:08.598397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 126/258] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:08.605148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.722293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.752892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917993e5-8477-4f9e-a636-9ec688904e15 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.802942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.803143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.803414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.803691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.804388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:08.804692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:08.806061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.819138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 133/259] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:08.820583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.825134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.961990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d96f2d-1e96-436e-ae04-144476124209 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.010066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.010709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.010993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.011270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] PATH_INFO: `/v3/users/11673db0a9f248858411797c26633075` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.016634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] RBAC: Authorizing `identity:get_user(user_id=11673db0a9f248858411797c26633075)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.018128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.020101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 127/260] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/users/11673db0a9f248858411797c26633075 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.026205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.075615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.222044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa07ad2-ca36-406a-aab4-7c615e0461d9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.251147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.289839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.290900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.292231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.293598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.310210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8addcd2-b627-44b5-9b86-4353221f4b8a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.316331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.376197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.377222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.380444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.380444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.398782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:09.403927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:09.404831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:09.405607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:09.408223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 128/261] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:09.416930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.460775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.462317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.626112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b9e4c-8e79-4be3-8e7e-180f03081eff None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.665342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 134/262] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:09.671125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.704310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.712826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.713570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.713861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.714139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.714826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:09.715132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.716573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.741387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 129/263] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.743166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.747965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.854371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa25940-36aa-4433-845f-bb05f6eb0c7a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.926893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.928204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.928537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.928817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.930977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.941056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.942860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 135/264] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:09.951388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.006628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583007-f487-4fa7-a0c1-f73f0468c47d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.050987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f36b2-cbe3-41d0-9883-397a9dbafdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.051631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f36b2-cbe3-41d0-9883-397a9dbafdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.051912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f36b2-cbe3-41d0-9883-397a9dbafdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.052741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f36b2-cbe3-41d0-9883-397a9dbafdc0 None admin] PATH_INFO: `/v3/users/11673db0a9f248858411797c26633075` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.053938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f36b2-cbe3-41d0-9883-397a9dbafdc0 None admin] RBAC: Authorizing `identity:get_user(user_id=11673db0a9f248858411797c26633075)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.055391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f36b2-cbe3-41d0-9883-397a9dbafdc0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.057344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 130/265] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/users/11673db0a9f248858411797c26633075 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:10.058526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d13e05-ad38-434b-97ef-f80c469ca01b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.058808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d13e05-ad38-434b-97ef-f80c469ca01b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.059083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d13e05-ad38-434b-97ef-f80c469ca01b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.065437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 131/266] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:10.066187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d13e05-ad38-434b-97ef-f80c469ca01b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.116898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8662be-23c5-4a36-a20d-de5a5110cd02 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.131038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.138429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.183127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d13e05-ad38-434b-97ef-f80c469ca01b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.220217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb997c3-5d38-4495-af5a-3660a43b03ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.221072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.221565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb997c3-5d38-4495-af5a-3660a43b03ea None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.221692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.221913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.222244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb997c3-5d38-4495-af5a-3660a43b03ea None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.222662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb997c3-5d38-4495-af5a-3660a43b03ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.222938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=11673db0a9f248858411797c26633075) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:10.223345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.225439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99f5b-7a2f-42a9-8597-61b9b0d260c1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.242577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 136/267] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/role_assignments?user.id=11673db0a9f248858411797c26633075&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:10.244223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0646c-11dd-4797-b9fe-f209803a6fbf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.244510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0646c-11dd-4797-b9fe-f209803a6fbf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.244907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0646c-11dd-4797-b9fe-f209803a6fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.293418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d7e0646c-11dd-4797-b9fe-f209803a6fbf None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:10.294787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d7e0646c-11dd-4797-b9fe-f209803a6fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:10.303541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb997c3-5d38-4495-af5a-3660a43b03ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.305426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb997c3-5d38-4495-af5a-3660a43b03ea None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.375445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.428165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 132/268] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:10.429540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a7429-2888-4bfa-be33-a3397ef43254 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.429823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a7429-2888-4bfa-be33-a3397ef43254 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.430102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a7429-2888-4bfa-be33-a3397ef43254 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.433692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 133/269] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:10.434356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a7429-2888-4bfa-be33-a3397ef43254 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.482298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 137/270] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:10.489382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5663c-cd8e-4026-819f-980e6cdc3f80 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.491906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5663c-cd8e-4026-819f-980e6cdc3f80 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.492451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5663c-cd8e-4026-819f-980e6cdc3f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.530519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-78b5663c-cd8e-4026-819f-980e6cdc3f80 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:10.533244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-78b5663c-cd8e-4026-819f-980e6cdc3f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:10.614754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a7429-2888-4bfa-be33-a3397ef43254 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.661036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3cd26-213f-4f05-93ac-d27d2c98eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.662625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3cd26-213f-4f05-93ac-d27d2c98eba4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.663153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3cd26-213f-4f05-93ac-d27d2c98eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.663617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3cd26-213f-4f05-93ac-d27d2c98eba4 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.667070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3cd26-213f-4f05-93ac-d27d2c98eba4 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.669559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3cd26-213f-4f05-93ac-d27d2c98eba4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.671774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 134/271] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:10.677300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a82a8-4772-46d0-94e9-f042e499fafd None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.677971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a82a8-4772-46d0-94e9-f042e499fafd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.678458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a82a8-4772-46d0-94e9-f042e499fafd None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.679857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 135/272] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:10.681915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a82a8-4772-46d0-94e9-f042e499fafd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.706420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 138/273] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:10.707532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.707980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.710005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.727161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.731033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:10.732623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:10.759278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.816798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a82a8-4772-46d0-94e9-f042e499fafd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.827149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 139/274] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:10.828984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.887475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.888191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.888474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.888741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.889475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] RBAC: Adding query filter params (role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=11673db0a9f248858411797c26633075) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:10.889827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.891263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.907806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 136/275] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/role_assignments?user.id=11673db0a9f248858411797c26633075&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=87236176cec04669b7f8d8199ce12228 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:10.947596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8457c-e2bd-42c6-b65d-8370ec34ab75 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.985355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.986229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.986673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.987090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.999585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.007785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:11.012774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:11.013678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:11.014801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 140/276] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:11.019995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.045653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.160076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.169477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f3bd1-b8b4-4c6b-b127-5c4848a64b2d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.172912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec03694-0fd9-49ea-a51a-2dc592bd7b31 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.210676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.211328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.211607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.211879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.213047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.214476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.216266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 137/277] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:11.217119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.219978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.219978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.219978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.219978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.220360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.222502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.222502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.236298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 141/278] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.237283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.331612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0009ae-b29c-455a-be45-72bb22ef157a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.331869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.352774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18731146-23c6-4cff-845d-e60f04106ae1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.370263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.370904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.371184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.371460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.382963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:11.388292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:11.389162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:11.390216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 138/279] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:11.391297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.391297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.391438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.391699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.392325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.393745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.394815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.418171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 142/280] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:11 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:11.419415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.432694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.539695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b278a29-65d8-4940-9481-167791c94aa4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.564192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afb644-612c-4395-83a2-8195e5159b48 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.581279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82221-a892-444d-bbff-7d12de1e8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.581978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82221-a892-444d-bbff-7d12de1e8310 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.582261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82221-a892-444d-bbff-7d12de1e8310 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.582538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82221-a892-444d-bbff-7d12de1e8310 None admin] PATH_INFO: `/v3/users/532342705bd5429392175cea46ba4e0b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.601585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82221-a892-444d-bbff-7d12de1e8310 None admin] RBAC: Authorizing `identity:get_user(user_id=532342705bd5429392175cea46ba4e0b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.602619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82221-a892-444d-bbff-7d12de1e8310 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.604322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 139/281] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/users/532342705bd5429392175cea46ba4e0b => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.604322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ef9a9-3b8b-4fa6-8155-f2d5d33cc16c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.604322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ef9a9-3b8b-4fa6-8155-f2d5d33cc16c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.604322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ef9a9-3b8b-4fa6-8155-f2d5d33cc16c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.604322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 140/282] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:11.609540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.610030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ef9a9-3b8b-4fa6-8155-f2d5d33cc16c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.610542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.610965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.611379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.612239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.612709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.614276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c87ff-97d3-4be6-8973-aa84416840c8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.619985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.625014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 143/283] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.626475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.627123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.627582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.650030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:11.651327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:11.714806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.737064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ef9a9-3b8b-4fa6-8155-f2d5d33cc16c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.752324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 144/284] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:11.753257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.779992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.780756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.781125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.781490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.782318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, user.id=532342705bd5429392175cea46ba4e0b, scope.system=all) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.783480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.785069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.791468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 141/285] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/role_assignments?user.id=532342705bd5429392175cea46ba4e0b&scope.system=all&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.885048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b017308-1812-442a-b657-d8a07ff079ae None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.929980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.936141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.936794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.937072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.937405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] PATH_INFO: `/v3/users/dafa9f6799db46b7a61b499b6df9d5c8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.944344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] RBAC: Authorizing `identity:get_user(user_id=dafa9f6799db46b7a61b499b6df9d5c8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.946084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.947240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 145/286] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/users/dafa9f6799db46b7a61b499b6df9d5c8 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.952853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.081823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.125116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36826ae-005b-489e-9a13-2dc7590ec576 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.197825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.199278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.199780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.200288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.204570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:12.206277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:12.208855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 142/287] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:12.216185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.218363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a69f8-2d70-4380-95ab-f8d788d2ab64 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.239883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:12.280664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.281704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.282212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.282683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.295368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:12.300994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:12.301882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:12.303352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:12.305727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 146/288] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:12.316034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.406283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702d02-a256-486d-8336-3df495d0d2ac None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.419983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:12.473919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.475595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.475885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.476190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/dafa9f6799db46b7a61b499b6df9d5c8/roles/7f0910382fdc47ad8396a6c442cda86d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.480912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6897c5d1cf24f76b49b423cc3b01774, user_id=dafa9f6799db46b7a61b499b6df9d5c8, role_id=7f0910382fdc47ad8396a6c442cda86d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:12.482378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:12.491611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f247a-99c4-4331-8a03-ff46144b6974 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.516982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 143/289] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:12 2022] PUT /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/users/dafa9f6799db46b7a61b499b6df9d5c8/roles/7f0910382fdc47ad8396a6c442cda86d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:26:12.535318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.535971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.536273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.536665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.538314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:12.538631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:12.540165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:12.549800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 147/290] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:12.556492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.668326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:12.708701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36db226-c111-47fd-a4cf-e0fc417810d2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.748454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.749090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.749371 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.749646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] PATH_INFO: `/v3/users/b26ec8597f7f47db996ed183a77d2842` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.752499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] RBAC: Authorizing `identity:get_user(user_id=b26ec8597f7f47db996ed183a77d2842)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:12.754011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:12.756770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 144/291] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:12 2022] GET /identity/v3/users/b26ec8597f7f47db996ed183a77d2842 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:12.762516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.873173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc16e9a-6fc3-439a-98e9-ee6158ca4068 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.900983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:12.922079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.922624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.922907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.923186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.014874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:13.016353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:13.159445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 148/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:13.169118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:13.305163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868afa0e-0e43-4cfc-9a9f-5491e6d73fa6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:13.351933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4123fe-6ad4-41e4-95f7-ad714926fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:13.352936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4123fe-6ad4-41e4-95f7-ad714926fb3d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.353418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4123fe-6ad4-41e4-95f7-ad714926fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.353913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4123fe-6ad4-41e4-95f7-ad714926fb3d None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.360721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4123fe-6ad4-41e4-95f7-ad714926fb3d None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:13.362414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4123fe-6ad4-41e4-95f7-ad714926fb3d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:13.365208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 145/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:13 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:13.371006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:13.406369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:13.537817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c07f0-a239-4abc-baa1-df481c96b71a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:13.582618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:13.583146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.583973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.584447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.585189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=b26ec8597f7f47db996ed183a77d2842) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:13.585493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:13.586916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2a67f-d472-4fee-ab86-2609d9d83a26 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:13.593427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 149/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:13 2022] GET /identity/v3/role_assignments?user.id=b26ec8597f7f47db996ed183a77d2842&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:14.070931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f2f3-0f5b-4fe6-b8cf-f13659300e05 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.071247 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f2f3-0f5b-4fe6-b8cf-f13659300e05 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.071491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f2f3-0f5b-4fe6-b8cf-f13659300e05 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.073116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 146/295] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:14.074069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247893-666c-4782-9f67-15fbf9f02ed2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.074833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247893-666c-4782-9f67-15fbf9f02ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.075112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247893-666c-4782-9f67-15fbf9f02ed2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.076753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 150/296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:14.082033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.082164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1ed0-d47b-4032-8786-ff62afa1be76 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.082359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.082466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1ed0-d47b-4032-8786-ff62afa1be76 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.082612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.082701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1ed0-d47b-4032-8786-ff62afa1be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.104855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e1fd1ed0-d47b-4032-8786-ff62afa1be76 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:14.106052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e1fd1ed0-d47b-4032-8786-ff62afa1be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:14.108391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:14.109712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:14.111796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.182203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 151/297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:14.183294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39abe1-83c9-4856-a91d-fdf41b04f300 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.183585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39abe1-83c9-4856-a91d-fdf41b04f300 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.183861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39abe1-83c9-4856-a91d-fdf41b04f300 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.185267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 152/298] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:14.190322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.190814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.191345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.208110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 147/299] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:14.217316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:14.218584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:14.228370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.302535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 153/300] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:14.454732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.500605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.563364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c42acf-5a23-40bf-80a6-c8600987421a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.601176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.601817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.602083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.602343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.611060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.616699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:14.622368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:14.623266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 148/301] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:14.628723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.642985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1d85a-82ca-457c-a195-e66c24eed1c5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.680744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.681382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.681666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.681940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.683038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.684510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.686121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 154/302] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:14.686959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.702503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.735123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e35167-4c9e-49f1-93f3-78f262a2cb57 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.773224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.773761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.774046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.774560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.785466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:14.791106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:14.792038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:14.793087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 149/303] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:14.794347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.801129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef08812-0fdf-4222-bf13-51cc7e6cc6a7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.838360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.838897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.839179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.839456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.840152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:14.840461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.841824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.854783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 155/304] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:14.854783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.862046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.912608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6915-613e-4c86-8453-49944be5d717 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.940970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.947880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.948518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.948798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.949068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] PATH_INFO: `/v3/groups` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.949690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.951129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.966655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 150/305] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Aug 24 16:26:14 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:14.966826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fedfd8-99ce-43fb-87b9-6d49497921b1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.967449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.010096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.010815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.011101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.011375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.012100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.012408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.013842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa13fbb-5409-4c20-a364-2ae7a0146b77 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.023823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 156/306] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:15.023823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4644-8f78-46b2-9dde-a58f63a2657c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.023823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4644-8f78-46b2-9dde-a58f63a2657c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.023823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4644-8f78-46b2-9dde-a58f63a2657c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.023823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 157/307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:15.032484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4644-8f78-46b2-9dde-a58f63a2657c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.105237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02f37d-b418-43e6-9119-827d794b9c60 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.126304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.148129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a8f35-d9bd-491d-b47e-9499f2d39ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.148764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a8f35-d9bd-491d-b47e-9499f2d39ecf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.149041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a8f35-d9bd-491d-b47e-9499f2d39ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.149313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a8f35-d9bd-491d-b47e-9499f2d39ecf None admin] PATH_INFO: `/v3/users/277bddb9840b4cbabe220a02deef459b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.155996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4644-8f78-46b2-9dde-a58f63a2657c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.164737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a8f35-d9bd-491d-b47e-9499f2d39ecf None admin] RBAC: Authorizing `identity:get_user(user_id=277bddb9840b4cbabe220a02deef459b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.166196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a8f35-d9bd-491d-b47e-9499f2d39ecf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.172918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 151/308] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/users/277bddb9840b4cbabe220a02deef459b => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:15.172918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.172918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.172918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.190677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:15.191931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:15.193496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.194481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.195112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.195390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] PATH_INFO: `/v3/users/532342705bd5429392175cea46ba4e0b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.197908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] RBAC: Authorizing `identity:get_user(user_id=532342705bd5429392175cea46ba4e0b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.200444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.207208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 158/309] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/users/532342705bd5429392175cea46ba4e0b => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:15.207208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.212377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.279975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 152/310] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:15.279975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.349714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364a935-5c1a-4476-98af-50d6ba3be81e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.394148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114be2bb-8422-483b-8950-b3e0f8fe7cec None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.399673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.400800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.401365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.401909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.403290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] RBAC: Adding query filter params (role.id=7f0910382fdc47ad8396a6c442cda86d, user.id=277bddb9840b4cbabe220a02deef459b, scope.system=all) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.403894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.404632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.406663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.415852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 159/311] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/role_assignments?user.id=277bddb9840b4cbabe220a02deef459b&scope.system=all&role.id=7f0910382fdc47ad8396a6c442cda86d => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:15.437179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.438156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.438574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.438988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] PATH_INFO: `/v3/system/users/532342705bd5429392175cea46ba4e0b/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.440823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=532342705bd5429392175cea46ba4e0b, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.442440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.454254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 153/312] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Aug 24 16:26:15 2022] PUT /identity/v3/system/users/532342705bd5429392175cea46ba4e0b/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:15.573338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.650416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.687861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25831e-00fe-464e-af9a-f99219d7e8ef None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.724091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.725269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.725556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.725879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.736323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:15.742820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:15.744508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 160/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:15.749998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.863029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37c32d-4cee-495b-ac1e-184b4bca1dae None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.900616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.901598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.902067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.902529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.903382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.903914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.905585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.919656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 154/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:15.920716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.039122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462a2e9-7f66-4b07-98c3-03169f1c6b2b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.070970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.080478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.081120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.081410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.081690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] PATH_INFO: `/v3/users/dafa9f6799db46b7a61b499b6df9d5c8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.082842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] RBAC: Authorizing `identity:get_user(user_id=dafa9f6799db46b7a61b499b6df9d5c8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.084488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.086417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 161/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/users/dafa9f6799db46b7a61b499b6df9d5c8 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:16.091815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.127984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.201311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cc40a-e009-4216-9549-afb5d0aa2c9e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.236025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.236843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.237261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.237670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.306918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.308643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.340323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.413074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 155/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:16.418897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.527350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb645b-60ac-4eab-b732-2d68044fd9c5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.566432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17d23-82cf-40a9-a2d3-16bd2ea5aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.567410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17d23-82cf-40a9-a2d3-16bd2ea5aec5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.568548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17d23-82cf-40a9-a2d3-16bd2ea5aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.569067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17d23-82cf-40a9-a2d3-16bd2ea5aec5 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.570534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17d23-82cf-40a9-a2d3-16bd2ea5aec5 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.572187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17d23-82cf-40a9-a2d3-16bd2ea5aec5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.574399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 162/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:16.580402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.689091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195342e9-0816-4e48-9d8c-0a6110e95afb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.725712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.732071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.732939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.733499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.733971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.735022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] RBAC: Adding query filter params (role.id=7f0910382fdc47ad8396a6c442cda86d, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774, user.id=dafa9f6799db46b7a61b499b6df9d5c8) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:16.735331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.736790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fe70c-a96a-4dfc-bf6e-ecfb996faa9e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.743319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 156/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/role_assignments?user.id=dafa9f6799db46b7a61b499b6df9d5c8&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=7f0910382fdc47ad8396a6c442cda86d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:16.961614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0614ab-4dd8-448f-bb99-bc19a76b3055 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.961981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0614ab-4dd8-448f-bb99-bc19a76b3055 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.962152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0614ab-4dd8-448f-bb99-bc19a76b3055 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.963628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 163/319] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:16.969256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43c040-036f-4a6c-9908-deb0b5d26157 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.970143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43c040-036f-4a6c-9908-deb0b5d26157 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.970417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43c040-036f-4a6c-9908-deb0b5d26157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.988941 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3d43c040-036f-4a6c-9908-deb0b5d26157 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:16.990221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3d43c040-036f-4a6c-9908-deb0b5d26157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:17.065311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 157/320] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:17.334133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0614ab-4dd8-448f-bb99-bc19a76b3055 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.399557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:17.414798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7844-ebc4-4394-be84-a76a3af93502 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.415104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7844-ebc4-4394-be84-a76a3af93502 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.415356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c7844-ebc4-4394-be84-a76a3af93502 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.416878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 158/321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:17.422031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193782-5b77-4fde-b7f8-7b99e72056c7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.422317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193782-5b77-4fde-b7f8-7b99e72056c7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.422595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193782-5b77-4fde-b7f8-7b99e72056c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.441370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9c193782-5b77-4fde-b7f8-7b99e72056c7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:17.442690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9c193782-5b77-4fde-b7f8-7b99e72056c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:17.444148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0614ab-4dd8-448f-bb99-bc19a76b3055 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.480640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.481273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.481558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.481840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.482939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:17.484433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.486161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 164/322] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:17.491634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.515027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 159/323] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:17.573985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3cabe-93bf-4bd2-96e2-f5625c402354 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.574535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3cabe-93bf-4bd2-96e2-f5625c402354 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.574814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3cabe-93bf-4bd2-96e2-f5625c402354 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.576335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 160/324] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:17.582633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.582920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.583198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.597990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2105e-d901-47d8-a8e3-505fe7f0143a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.601375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:17.602551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:17.634882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:17.640608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.641219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.641517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.641771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] PATH_INFO: `/v3/groups` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.642836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:17.643817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.660182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:17.671973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 165/325] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Aug 24 16:26:17 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:26:17.692129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 161/326] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:17.797101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.904118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b9bdf-5ece-4850-b0c2-e0ebaa4e7635 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.943834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.944553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.944837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.945114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.955742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:17.961695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:17.962577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:17.963349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 166/327] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:17.965561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.970120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.042373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:18.111235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8651f-05da-49d2-925c-de1a8f838039 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.119381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e25ae8-26c4-4f7f-b951-81e251652f85 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.157248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.158221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.158937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.159701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.160572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:18.161022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.163669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.164249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.164964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.165314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.165659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.174749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 167/328] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:18.180646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.183306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:18.188433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:18.189306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:18.190079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:18.192537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 162/329] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:18.200993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.259137 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:18.332092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6159996-0c1a-47c2-a2f0-ffd7b742f9eb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.332850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1022428-cb96-447c-8f37-dd1b69324e25 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.352790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:18.371977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.372916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.373422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.373913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.374883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:18.375387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.377163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.381455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.382423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.382905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.383378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] PATH_INFO: `/v3/users/277bddb9840b4cbabe220a02deef459b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.385145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] RBAC: Authorizing `identity:get_user(user_id=277bddb9840b4cbabe220a02deef459b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.387181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.388796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 163/330] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:18.389546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 168/331] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/users/277bddb9840b4cbabe220a02deef459b => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:18.398249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.401022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.523070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48240b0-faa2-4868-8323-13b321722786 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.523663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f9c7b-de27-40f7-8c3d-7d5cd6969c90 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.533448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:18.563925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.564331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0badb-da61-48a2-b645-d48e48234ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.564582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.564873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.565151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] PATH_INFO: `/v3/users/532342705bd5429392175cea46ba4e0b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.565916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0badb-da61-48a2-b645-d48e48234ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.566370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] RBAC: Authorizing `identity:get_user(user_id=532342705bd5429392175cea46ba4e0b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.566604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0badb-da61-48a2-b645-d48e48234ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.567068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0badb-da61-48a2-b645-d48e48234ef2 None admin] PATH_INFO: `/v3/system/users/277bddb9840b4cbabe220a02deef459b/roles/7f0910382fdc47ad8396a6c442cda86d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.568381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.570052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0badb-da61-48a2-b645-d48e48234ef2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=277bddb9840b4cbabe220a02deef459b, role_id=7f0910382fdc47ad8396a6c442cda86d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.570358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 164/332] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/users/532342705bd5429392175cea46ba4e0b => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:18.576428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0badb-da61-48a2-b645-d48e48234ef2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.579078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.602185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 169/333] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Aug 24 16:26:18 2022] PUT /identity/v3/system/users/277bddb9840b4cbabe220a02deef459b/roles/7f0910382fdc47ad8396a6c442cda86d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:26:18.718982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfae0e-6664-4938-a2ca-3be32906d484 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.757715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.758756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.759251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.759720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.760696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] RBAC: Adding query filter params (role.id=e1f816c1dc84469786d5a59d7920b1f1, user.id=532342705bd5429392175cea46ba4e0b, scope.system=all) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:18.761369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.763043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3738c96-26f8-4e34-8e74-4c4b6a8acb4f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.778031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 165/334] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/role_assignments?user.id=532342705bd5429392175cea46ba4e0b&scope.system=all&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:19.902644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d679-c6d3-4749-a834-ca436f32f992 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.903600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d679-c6d3-4749-a834-ca436f32f992 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.904146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286d679-c6d3-4749-a834-ca436f32f992 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.905959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 170/335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:19.910996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e740-2594-45ba-b581-d62ac38f59d1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.912547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e740-2594-45ba-b581-d62ac38f59d1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.913033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e740-2594-45ba-b581-d62ac38f59d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.936565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7ef2e740-2594-45ba-b581-d62ac38f59d1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:19.938084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7ef2e740-2594-45ba-b581-d62ac38f59d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:19.977046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.027452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 166/336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:20.433575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5e5c7-cf4b-42c7-a5c3-a61ca49a0bf3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.434307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5e5c7-cf4b-42c7-a5c3-a61ca49a0bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.434811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5e5c7-cf4b-42c7-a5c3-a61ca49a0bf3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.436441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 171/337] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:20.441669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.442176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.443236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.465751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:20.467258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:20.548772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 167/338] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:20.586437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5e5c7-cf4b-42c7-a5c3-a61ca49a0bf3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.662500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.741355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5e5c7-cf4b-42c7-a5c3-a61ca49a0bf3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.782283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.782918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.783188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.783469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.797383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:20.803463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:20.804391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:20.805164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 172/339] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:20.810353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.925474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.944720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.950439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937774-2f6a-45de-aded-68d26d7448e8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.018135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.018682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.018970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.019244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.019923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:21.020367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.022040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.043820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 168/340] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:21.050083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.120230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf2297-27d8-4461-8844-95b9abb09f42 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.136891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.194301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a31bb4-4710-45f2-aff1-d2ae690e3a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.195329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a31bb4-4710-45f2-aff1-d2ae690e3a47 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.195771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a31bb4-4710-45f2-aff1-d2ae690e3a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.196258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a31bb4-4710-45f2-aff1-d2ae690e3a47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.204389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb869bcb-43dc-4396-ab5f-ebaa7aab43b1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.222870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a31bb4-4710-45f2-aff1-d2ae690e3a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-49a31bb4-4710-45f2-aff1-d2ae690e3a47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:21.229201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:21.230297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:21.231297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:21.233631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 173/341] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:21.240169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53419d-2dc1-41a4-ae8d-3ad0604e5433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.241202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0b5bc-fa08-4f39-97a1-d1bba399851e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.241391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53419d-2dc1-41a4-ae8d-3ad0604e5433 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.241676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53419d-2dc1-41a4-ae8d-3ad0604e5433 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.241930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0b5bc-fa08-4f39-97a1-d1bba399851e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.242057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53419d-2dc1-41a4-ae8d-3ad0604e5433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.242464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0b5bc-fa08-4f39-97a1-d1bba399851e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.244243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 174/342] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:21.245211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0b5bc-fa08-4f39-97a1-d1bba399851e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.272750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.321107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53419d-2dc1-41a4-ae8d-3ad0604e5433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.322609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53419d-2dc1-41a4-ae8d-3ad0604e5433 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.423698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0b5bc-fa08-4f39-97a1-d1bba399851e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.438652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 169/343] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:21.439733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.440055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.440334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.473161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:21.474720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:21.484653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.485627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.486103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.486581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.487528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:21.488058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.490157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70835a84-0eec-4d7b-8ac0-af640b2a63f1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.499539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 175/344] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:21.501092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e146a75-e751-4cb7-be8a-475fb68836a8 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.501569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e146a75-e751-4cb7-be8a-475fb68836a8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.502050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e146a75-e751-4cb7-be8a-475fb68836a8 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.503424 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 176/345] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:21.504344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e146a75-e751-4cb7-be8a-475fb68836a8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.539806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.557187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.565715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 170/346] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:21.566951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.663642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e146a75-e751-4cb7-be8a-475fb68836a8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.740729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eee7c-21de-41a8-b8be-65d59891a567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.743361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eee7c-21de-41a8-b8be-65d59891a567 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.745094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eee7c-21de-41a8-b8be-65d59891a567 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.745564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eee7c-21de-41a8-b8be-65d59891a567 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.747780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36eee7c-21de-41a8-b8be-65d59891a567 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.750737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36eee7c-21de-41a8-b8be-65d59891a567 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.757119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 177/347] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:21 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:21.759600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.761001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.761469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.767379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e590e38-7387-4814-9850-b9c5be71496d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.799973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:21.801109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:21.818795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.819856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.820352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.820862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.883394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.899222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 178/348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:21.899865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.900464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.902171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.929096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.013910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 171/349] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:22.015397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.017135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f367e5-dbfa-4f86-aea7-fab54f024259 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.057259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.057833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.058113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.058385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.065227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.066732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.069437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 179/350] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:21 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:22.070239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.110230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.127813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa564e0c-2b97-43e7-881a-c965a32692d3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.165248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.165940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.166215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.166483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.168769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.178961 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:22.179254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e18ae-5097-4611-9f29-4e278558588b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:22.184604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:22.185535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 172/351] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:22.187709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.216747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.217392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.217676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.217950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.219105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.220594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.222543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 180/352] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:22.223318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.311489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76a3cc-94ea-45c4-a87d-5c0e4a6ff62a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.342711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd984de-4067-4dde-83de-e1516e321f8a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.354899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.355512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.355938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.356236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.356971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=5711ce46bc304c3d9d171989b92e55bb) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:22.357275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.358906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.365244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 173/353] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:22.366262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.381202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.382878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.385241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.385241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.385241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.388474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:22.388474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.388474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.403184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 181/354] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:22.409275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.451374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.523539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed829c-2928-45ee-9b45-09e105b27fab None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.591195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.592826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.593115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.593399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.594488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79605ceb-5228-4c4d-8157-8000392f5f5e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.597299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.601016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.603019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 174/355] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:22.603645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.649514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f9ec5-7d92-44b5-8a18-d744a2f63450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.651247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f9ec5-7d92-44b5-8a18-d744a2f63450 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.651691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f9ec5-7d92-44b5-8a18-d744a2f63450 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.652150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f9ec5-7d92-44b5-8a18-d744a2f63450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.724175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.760006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8823219-62d1-4e29-8c99-ee95b41f5a87 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.767552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f9ec5-7d92-44b5-8a18-d744a2f63450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.769104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f9ec5-7d92-44b5-8a18-d744a2f63450 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.809850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.821486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01586f-c152-4050-87eb-0867c87ba1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.823495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01586f-c152-4050-87eb-0867c87ba1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.824117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01586f-c152-4050-87eb-0867c87ba1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.825090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01586f-c152-4050-87eb-0867c87ba1b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.842419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01586f-c152-4050-87eb-0867c87ba1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5c01586f-c152-4050-87eb-0867c87ba1b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:22.850420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:22.851396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:26:22.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:22.853491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 175/356] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:22.855504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84e4c-b771-471b-b731-f4a637fe5587 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.855813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84e4c-b771-471b-b731-f4a637fe5587 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.856098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84e4c-b771-471b-b731-f4a637fe5587 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.859095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 176/357] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:22.860321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84e4c-b771-471b-b731-f4a637fe5587 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.953231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 182/358] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:22.953231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154273e-c931-4f3b-ac0c-6a6ff95471bf None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.953231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154273e-c931-4f3b-ac0c-6a6ff95471bf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.953231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154273e-c931-4f3b-ac0c-6a6ff95471bf None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.953231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 183/359] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:22.953979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154273e-c931-4f3b-ac0c-6a6ff95471bf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.007328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84e4c-b771-471b-b731-f4a637fe5587 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.062159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.063443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.063863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.064314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.065250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=5711ce46bc304c3d9d171989b92e55bb) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:23.065566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.067732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fedb29-3e36-4f3f-9526-7bb75866f397 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.079117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 177/360] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=87236176cec04669b7f8d8199ce12228 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.081393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.081712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.081995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.102381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.108181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:23.109444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:23.139969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154273e-c931-4f3b-ac0c-6a6ff95471bf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.178601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.207278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.208729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.209018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.209292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.210289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:23.210602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.212190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2cc4-b57f-4fb9-bc34-cb7af80b336a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.222991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 178/361] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:23.224837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 184/362] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.226680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.227254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.227662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.228923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.250191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:23.254023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:23.351151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 185/363] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:23.352345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.396757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84020e03-0470-4fd3-bb65-914b096af07e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.451254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.461299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.461951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.462232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.462502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.465189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.466645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.468320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 179/364] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.487755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.592804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1001649-e603-4bc5-9f86-7396743587c6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.623960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.662000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420abd0-ff7c-4fb2-a586-b8cfb0388681 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.665033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.665677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.665954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.666227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] PATH_INFO: `/v3/users/277bddb9840b4cbabe220a02deef459b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.667362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] RBAC: Authorizing `identity:get_user(user_id=277bddb9840b4cbabe220a02deef459b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.669078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.670860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 186/365] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/users/277bddb9840b4cbabe220a02deef459b => generated 293 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.671401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.727566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.728242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.728517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.728790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.730700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.732167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.733973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 180/366] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:23.734528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.795070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d13deb-0b52-469e-9bdb-79eaf2fe627e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.836052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.836697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.837376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.837657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.851369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:23.857506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:23.858417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 187/367] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:23.859784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.862156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.908191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded64c39-0cd6-4994-bfb5-8b5367f0dd38 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.923699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.954885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.955499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.955779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.956080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.957341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] RBAC: Adding query filter params (role.id=7f0910382fdc47ad8396a6c442cda86d, user.id=277bddb9840b4cbabe220a02deef459b, scope.system=all) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:23.957665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.960044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.976173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 181/368] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/role_assignments?user.id=277bddb9840b4cbabe220a02deef459b&scope.system=all&role.id=7f0910382fdc47ad8396a6c442cda86d => generated 510 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.980248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.014093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98702f52-4c42-44e7-a91b-e029238ba84f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.064657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.065432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.065732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.066068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.066815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.067130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.068888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.078253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 188/369] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:24.079188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.227639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6caf9-f5da-4569-8bba-af30d089b5e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.287384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c83bcc-cd6e-4539-9b22-30a397d4f70f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.296345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.297896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.322317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.323323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.323843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.324620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.344746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:24.354603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:24.355567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:24.356774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:24.362636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 182/370] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:24.366494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.379824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.381517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.381998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.382531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.383472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.383991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.385655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.399679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 189/371] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:24.405451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.555936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4de5-00d5-4b27-ae6d-bdbff4978a23 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.557200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2ba22-8586-4584-9e04-5d25dba63af3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.615500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.616154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.616437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.616713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.624806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.625438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.625716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.625993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.626681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.626987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.628426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.651040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 183/372] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:24.660972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.686922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.727813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.729385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.794000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.836349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 190/373] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:24.841942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.920307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.953250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37838185-2db4-46de-8eeb-090a75026808 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.959028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef514b-5066-4194-acf2-eeeeebdf5e70 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.991354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.992025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.992310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.992590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.993748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.995173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.997243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 191/374] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:25.003184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.065035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cba7a7-c774-4a8c-b186-5e6ae215bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.065675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cba7a7-c774-4a8c-b186-5e6ae215bec5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.066132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cba7a7-c774-4a8c-b186-5e6ae215bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.066415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cba7a7-c774-4a8c-b186-5e6ae215bec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.133280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24fea7-2dff-4897-b9ed-e18d91018ec1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.172422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cba7a7-c774-4a8c-b186-5e6ae215bec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.173905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cba7a7-c774-4a8c-b186-5e6ae215bec5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.183233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c310c54-5f80-4c87-aa1b-b39dee89b87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.184320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c310c54-5f80-4c87-aa1b-b39dee89b87d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.184819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c310c54-5f80-4c87-aa1b-b39dee89b87d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.185881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c310c54-5f80-4c87-aa1b-b39dee89b87d None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.188684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c310c54-5f80-4c87-aa1b-b39dee89b87d None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.190386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c310c54-5f80-4c87-aa1b-b39dee89b87d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.192770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 192/375] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:25.198898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca270-ab8d-406b-8cbf-0cfae422fbe7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.199587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca270-ab8d-406b-8cbf-0cfae422fbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.200118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca270-ab8d-406b-8cbf-0cfae422fbe7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.201683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 193/376] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:25.202618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca270-ab8d-406b-8cbf-0cfae422fbe7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.208292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.237666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.314123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 184/377] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:25.315386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.315675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.320036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.337218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca270-ab8d-406b-8cbf-0cfae422fbe7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.338898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:25.341123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:25.379729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.380927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.381433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.381959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.382886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:25.383467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.385176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.392887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 194/378] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=87236176cec04669b7f8d8199ce12228 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:25.398546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.441118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 185/379] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:25.516510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.574536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c94d80-b84d-47b5-afe3-53e10de515f2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.618496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec494a0d-b700-4d55-ac7e-c0cb08a39a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.619135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec494a0d-b700-4d55-ac7e-c0cb08a39a26 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.619415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec494a0d-b700-4d55-ac7e-c0cb08a39a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.619686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec494a0d-b700-4d55-ac7e-c0cb08a39a26 None admin] PATH_INFO: `/v3/projects/ad742c0425e4471ab69af90dda50deea` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.621629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec494a0d-b700-4d55-ac7e-c0cb08a39a26 None admin] RBAC: Authorizing `identity:get_project(project_id=ad742c0425e4471ab69af90dda50deea)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.623070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec494a0d-b700-4d55-ac7e-c0cb08a39a26 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.624756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 195/380] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/projects/ad742c0425e4471ab69af90dda50deea => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:25.631874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.646961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9413-94bf-498e-948b-7865c65ddce1 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.647231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9413-94bf-498e-948b-7865c65ddce1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.647417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9413-94bf-498e-948b-7865c65ddce1 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.649055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 196/381] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:25.665761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.666096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.666320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.695649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:25.696879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:25.737592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.768738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69e65d-184a-42ee-9bb5-30355ff6764f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.807627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.808309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.808585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.808857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] PATH_INFO: `/v3/groups/c383187e76f7455fa9c6f6d63f37054a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.815066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] RBAC: Authorizing `identity:get_group(group_id=c383187e76f7455fa9c6f6d63f37054a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.818597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.821577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 186/382] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/groups/c383187e76f7455fa9c6f6d63f37054a => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:25.823755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 197/383] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:25.829064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.871249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.958513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aca0e-2452-4ff8-a949-bb64f97623ad None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.977004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.001103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.001103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.001103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.001103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.001715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] RBAC: Adding query filter params (group.id=c383187e76f7455fa9c6f6d63f37054a, role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=ad742c0425e4471ab69af90dda50deea) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.001715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.001715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.006799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 187/384] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/role_assignments?group.id=c383187e76f7455fa9c6f6d63f37054a&scope.project.id=ad742c0425e4471ab69af90dda50deea&role.id=580c9e29995a41c49a5a943d7b48532c => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.044785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4f5dc-5f00-4a94-97d3-7414160c1dd7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.102208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.102876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.103162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.103437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.120409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:26.125417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:26.126475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:26.127705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 198/385] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:26.133172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.148411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.265809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b68c3-eaf2-49e7-9167-fad8981a6d0c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.306452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.307088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.307375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.307653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.308371 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.308681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.310076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.320698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 188/386] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.326266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.337250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.483686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.558456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8eb92-894e-4323-bb17-d817725135bd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.572816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16475452-cdbb-4514-93fe-0cf64110261f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.605827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.628560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.629198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.629474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.629745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.631612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.633100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.636337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 199/387] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.646202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.653525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.655096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.655385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.655664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.672327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:26.679821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:26.680771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:26.681568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:26.686377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 189/388] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:26.692417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.834927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0104-56c5-4218-906a-071d0e21a1e7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.863804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.878408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.879354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.880672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.882232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.883297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.884934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.886818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.903141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e24fd-6abe-4988-abcd-8f6404803a9b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.905324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 190/389] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:26.913294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.968643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0619f3-62da-4983-8b67-6db74c01602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.969327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0619f3-62da-4983-8b67-6db74c01602e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.969544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0619f3-62da-4983-8b67-6db74c01602e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.969816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0619f3-62da-4983-8b67-6db74c01602e None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.970994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0619f3-62da-4983-8b67-6db74c01602e None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.972561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0619f3-62da-4983-8b67-6db74c01602e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.975264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 200/390] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.976265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd8bd4-a3db-4a54-b70b-27f39b60cfb5 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.976544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd8bd4-a3db-4a54-b70b-27f39b60cfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.976811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd8bd4-a3db-4a54-b70b-27f39b60cfb5 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.978036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 201/391] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:26.983357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.983636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.983911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.006899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:27.008145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:27.042573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53da9c1-e627-4f7e-9841-195b6f0002a5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.086075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:27.086745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.087258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.087609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.088862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:27.089700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:27.090915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:27.092786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 191/392] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:27.093591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:27.111410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 202/393] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:27.112507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:27.174726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:27.238977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb60e69-0177-47b7-8c46-feaddd7a7e41 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.275119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61374e5-7bac-4eba-aa4b-ced0fbd9a6c8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.282448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:27.283771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.284299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.284774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb/groups/e6259141767841869cef233a128d3f51/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.288311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] RBAC: Authorizing `identity:create_grant(project_id=5711ce46bc304c3d9d171989b92e55bb, group_id=e6259141767841869cef233a128d3f51, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:27.290034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:27.310612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 192/394] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 16:26:27 2022] PUT /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb/groups/e6259141767841869cef233a128d3f51/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:27.326167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:27.326836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.327394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.327394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.328817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:27.330533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:27.332774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 203/395] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:27 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:27.339451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:27.497393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9c0df-c9e1-4f00-a6b5-ee55a7bb058f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.521315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:27.536936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a256fcb-a333-4f63-95b4-306cef69e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:27.537777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a256fcb-a333-4f63-95b4-306cef69e62e None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.538066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a256fcb-a333-4f63-95b4-306cef69e62e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.538346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a256fcb-a333-4f63-95b4-306cef69e62e None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb/groups/e6259141767841869cef233a128d3f51/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.541707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a256fcb-a333-4f63-95b4-306cef69e62e None admin] RBAC: Authorizing `identity:create_grant(project_id=5711ce46bc304c3d9d171989b92e55bb, group_id=e6259141767841869cef233a128d3f51, role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:27.543435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a256fcb-a333-4f63-95b4-306cef69e62e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:27.558335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 193/396] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 16:26:27 2022] PUT /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb/groups/e6259141767841869cef233a128d3f51/roles/87236176cec04669b7f8d8199ce12228 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:27.628923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:27.713167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:27.733784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9d590-0f5e-487d-b935-5fb8493f7ec7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.734424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9d590-0f5e-487d-b935-5fb8493f7ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.734903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9d590-0f5e-487d-b935-5fb8493f7ec7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.737347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 194/397] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:27.744671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.745235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.745844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.764190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dea54c-3735-40d0-a293-996a95649a04 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.780365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:27.781588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:27.807741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:27.808422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.808708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.808985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.820585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:27.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:27.826426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:27.827220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 204/398] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:27.833101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:27.869269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 195/399] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:27.958667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f1fca-2784-4a53-8a2e-6dbde71b426a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.981785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:28.003028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.003653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.003936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.004295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.005578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:28.006174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.007571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.016981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 205/400] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:28.022528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.158711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3ddf0-5325-435b-a692-6475d4c5e411 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.174075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:28.209078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.209674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.209960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.210232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.211400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.212862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.214782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 196/401] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:28.221216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.232384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.354013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db901a-c25b-43fb-917d-4bfa8f286a6d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.393353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.394040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.394320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.394595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.408436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:28.415983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:28.416913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:28.417944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 197/402] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:28.422400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.475110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:28.522385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b79bc5-2f12-4f7b-a18d-d3f0940f1573 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.537631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c016e6c-d285-42fe-b9b7-e9538bbc1db8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.577200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.577820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.578105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.578378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.579050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:28.579354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.580779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.591011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 198/403] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:28.596563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.635843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.636881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.637374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.637840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.639174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.640856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.642961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 206/404] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:28.648387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.712337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:28.717484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951e5ce-044b-4fff-826e-f60ff26676e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.744044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:28.759823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6a49f-eb73-44df-9f04-b529813ace95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.760952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a49f-eb73-44df-9f04-b529813ace95 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.761401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a49f-eb73-44df-9f04-b529813ace95 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.761823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a49f-eb73-44df-9f04-b529813ace95 None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.763188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6a49f-eb73-44df-9f04-b529813ace95 None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.764913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6a49f-eb73-44df-9f04-b529813ace95 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.767026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 199/405] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:28.768541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601333-5b4e-46eb-b3a3-3fdbfc4a5f0c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.768961 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601333-5b4e-46eb-b3a3-3fdbfc4a5f0c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.769368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601333-5b4e-46eb-b3a3-3fdbfc4a5f0c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.770989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 200/406] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:28.776638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c601333-5b4e-46eb-b3a3-3fdbfc4a5f0c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.805287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992c559-9386-4fc0-937e-ae5a76da39e4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.841036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abec13-c130-47b9-8742-43e7f943bf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.841885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abec13-c130-47b9-8742-43e7f943bf94 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.842222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abec13-c130-47b9-8742-43e7f943bf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.842614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abec13-c130-47b9-8742-43e7f943bf94 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/groups/e6259141767841869cef233a128d3f51/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.844620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abec13-c130-47b9-8742-43e7f943bf94 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6897c5d1cf24f76b49b423cc3b01774, group_id=e6259141767841869cef233a128d3f51, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.846146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abec13-c130-47b9-8742-43e7f943bf94 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.863500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 207/407] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 16:26:28 2022] PUT /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/groups/e6259141767841869cef233a128d3f51/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:28.865864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.866295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.866662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.891423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:28.892910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:28.906491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c601333-5b4e-46eb-b3a3-3fdbfc4a5f0c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:28.946965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:28.947837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.948328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.948860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.950283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:28.952076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:28.954335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 201/408] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:28.960696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.999513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 208/409] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:29.003164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.077592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4c6b4-7ec7-40bb-bfb2-d616d5c7260c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.113710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.115397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.115978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.116466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/groups/e6259141767841869cef233a128d3f51/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.118871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] RBAC: Authorizing `identity:create_grant(project_id=c6897c5d1cf24f76b49b423cc3b01774, group_id=e6259141767841869cef233a128d3f51, role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.120533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.135718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 202/410] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 16:26:28 2022] PUT /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774/groups/e6259141767841869cef233a128d3f51/roles/87236176cec04669b7f8d8199ce12228 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:29.328074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.373524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.449568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8aca2-f637-406a-84f7-9f76c1028181 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.494045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.494689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.494964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.495234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.511164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:29.515920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:29.516778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:29.517664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 209/411] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:29.524421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.665619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.672276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefe331-fbd3-421d-bee7-9ec9611d07cf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.721984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.722593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.722872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.723142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.724075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:29.724446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.725856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31524bd4-5420-4b6d-821b-c0d2d9d4fa3f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.734915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 203/412] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:29.740939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.858400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.861969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c959586-da0b-4b00-a3e6-8eb3d6db4e5d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.904022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc7915-47a4-46ba-844f-11beb198f1bc None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.904339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc7915-47a4-46ba-844f-11beb198f1bc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.904587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc7915-47a4-46ba-844f-11beb198f1bc None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.907133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 204/413] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:29.912254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.912641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.913018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.914046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.914688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.914969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.915242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] PATH_INFO: `/v3/groups/c383187e76f7455fa9c6f6d63f37054a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.921640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] RBAC: Authorizing `identity:get_group(group_id=c383187e76f7455fa9c6f6d63f37054a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.923037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.925003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 210/414] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/groups/c383187e76f7455fa9c6f6d63f37054a => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:29.930400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.932193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:29.933370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:30.056197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 205/415] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:30.087097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f38ca-91af-4267-b530-a8eb4d7a4249 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.121240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c723-67ba-4b84-ba9a-6a7fa0700d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.121872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c723-67ba-4b84-ba9a-6a7fa0700d20 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.122147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c723-67ba-4b84-ba9a-6a7fa0700d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.122417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c723-67ba-4b84-ba9a-6a7fa0700d20 None admin] PATH_INFO: `/v3/projects/ad742c0425e4471ab69af90dda50deea` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.123646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865c723-67ba-4b84-ba9a-6a7fa0700d20 None admin] RBAC: Authorizing `identity:get_project(project_id=ad742c0425e4471ab69af90dda50deea)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.125218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865c723-67ba-4b84-ba9a-6a7fa0700d20 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.127122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 211/416] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/projects/ad742c0425e4471ab69af90dda50deea => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:30.132931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.259329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:30.280187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8624c-5b98-4393-bfa3-18a247b1c924 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.308568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d3be2-0fd9-4f66-a597-3d98d3f00075 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.308857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d3be2-0fd9-4f66-a597-3d98d3f00075 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.309119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d3be2-0fd9-4f66-a597-3d98d3f00075 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.310694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 212/417] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:30.315749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.316064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.316461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.325837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.326521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.326802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.327077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] PATH_INFO: `/v3/projects/ad742c0425e4471ab69af90dda50deea/groups/c383187e76f7455fa9c6f6d63f37054a/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.332348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] RBAC: Authorizing `identity:create_grant(project_id=ad742c0425e4471ab69af90dda50deea, group_id=c383187e76f7455fa9c6f6d63f37054a, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.334506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.338009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:30.339296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:30.369272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 206/418] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 16:26:30 2022] PUT /identity/v3/projects/ad742c0425e4471ab69af90dda50deea/groups/c383187e76f7455fa9c6f6d63f37054a/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:30.370482 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.372300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:30.455717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 213/419] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:30.534786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e451b39-1c71-48c7-a02d-596164fa45ec None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.570615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.571248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.571531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.571803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.583933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:30.588936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:30.589828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:30.590621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:30.594823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 207/420] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:30.596176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.713328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66ac03-9999-44c8-82df-7fed01e6ed47 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.762681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.763305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.763576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.763856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.764569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:30.764870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.766422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.766539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.780957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 214/421] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:30.784331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:30.791835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.822879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:30.907791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a3a89-0db9-42ea-b24f-a00c37d241db None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.948083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dfbc5-204c-4f88-9f45-c5c380cb5c21 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.960603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae4008-4e45-4639-b253-b95b5aa46a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.961230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae4008-4e45-4639-b253-b95b5aa46a77 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.961512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae4008-4e45-4639-b253-b95b5aa46a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.961789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae4008-4e45-4639-b253-b95b5aa46a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.995118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.997795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.998079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.999933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.018491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:31.025432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:31.026319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:31.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 208/422] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:31.039933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.110327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae4008-4e45-4639-b253-b95b5aa46a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.111894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae4008-4e45-4639-b253-b95b5aa46a77 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.140253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.178399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b88b6-f258-4279-84d7-3e9c2ef13df9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.205594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.216720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.217335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.217618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.217894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.218575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:31.218884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.220303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.229883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 209/423] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:31.234993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.353310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e7b5-47df-435d-955f-951207a5a77e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.386522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.398264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc7ee5-950a-4b4f-bc58-2655dc113a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.398956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc7ee5-950a-4b4f-bc58-2655dc113a92 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.399235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc7ee5-950a-4b4f-bc58-2655dc113a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.399512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc7ee5-950a-4b4f-bc58-2655dc113a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.416825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 215/424] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:31.418372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c768eb-1dfe-4484-acae-e4448e112c1e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.418659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c768eb-1dfe-4484-acae-e4448e112c1e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.418939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c768eb-1dfe-4484-acae-e4448e112c1e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.420479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 216/425] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:31.422157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c768eb-1dfe-4484-acae-e4448e112c1e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.488152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc7ee5-950a-4b4f-bc58-2655dc113a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.490205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc7ee5-950a-4b4f-bc58-2655dc113a92 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.553091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c768eb-1dfe-4484-acae-e4448e112c1e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.592900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.602891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.603622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.603986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.604330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.605540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.607120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.609616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 210/426] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:31.610869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.611154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.611428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.616085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 217/427] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:31.618660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.634105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:31.635284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:31.659045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.716324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 211/428] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:31.717276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.753098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421638d-2b17-4eaf-a6da-2db08ffcf55a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.790710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936b6d-d461-498a-8137-97b3f200bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.791406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936b6d-d461-498a-8137-97b3f200bd73 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.791662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936b6d-d461-498a-8137-97b3f200bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.791967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936b6d-d461-498a-8137-97b3f200bd73 None admin] PATH_INFO: `/v3/projects/5711ce46bc304c3d9d171989b92e55bb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.793373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02936b6d-d461-498a-8137-97b3f200bd73 None admin] RBAC: Authorizing `identity:get_project(project_id=5711ce46bc304c3d9d171989b92e55bb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.794822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02936b6d-d461-498a-8137-97b3f200bd73 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.796832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 218/429] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/projects/5711ce46bc304c3d9d171989b92e55bb => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:31.797997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d27a-94af-49a0-95b7-333896be2346 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.798278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d27a-94af-49a0-95b7-333896be2346 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.798618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d27a-94af-49a0-95b7-333896be2346 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.800073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 219/430] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:31.801964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d27a-94af-49a0-95b7-333896be2346 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.863672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b0f9a-3149-47b4-956f-991edd665b4d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.876011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.903648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388386f4-4a02-4214-bab8-029a00082d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.904311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388386f4-4a02-4214-bab8-029a00082d5b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.904592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388386f4-4a02-4214-bab8-029a00082d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.904969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388386f4-4a02-4214-bab8-029a00082d5b None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.906094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388386f4-4a02-4214-bab8-029a00082d5b None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.907542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388386f4-4a02-4214-bab8-029a00082d5b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.909410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 212/431] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:31.910303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.910583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.910902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.931522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:31.932016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d27a-94af-49a0-95b7-333896be2346 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.932735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:31.957879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.971971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.972615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.972936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.973206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.974958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.976495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.978372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 220/432] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:31.979311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.019290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 213/433] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:32.020151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.104094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df643c5c-1902-4b0b-9d6b-1813a558680c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.136461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.143527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.144338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.144789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.145226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.146636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=5711ce46bc304c3d9d171989b92e55bb) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:32.147082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.148083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86076e04-ab38-4c98-9723-a7ca0860627f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.149000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.162869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 221/434] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:31 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:32.164098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.186707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.187619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.188057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.188428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.189370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=5711ce46bc304c3d9d171989b92e55bb) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:32.189776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.191430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.202918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 214/435] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=5711ce46bc304c3d9d171989b92e55bb&role.id=87236176cec04669b7f8d8199ce12228 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:32.312319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ac3d-548f-4c25-9bd2-8ebf9ab3f790 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.328952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.354363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.366927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.367595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.367983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.368269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.385375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:32.396047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:32.397243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:26:32.398099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:32.401747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 222/436] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:32.407800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.454668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc62552-55bc-4f26-bd11-9c1e45434647 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.475404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.496803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.497496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.497774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.498041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.513382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:32.521167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:32.522085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:32.522846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 215/437] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:32.528347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.537413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967e80a-db14-48f4-b818-11e4bed028a6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.585673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.586455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.586736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.587474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.588195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:32.588498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.589932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.601642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 223/438] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:32.607379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.631788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.656969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caf479-b62d-4c19-b9a1-b23762dd54cc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.705266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.705796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.706071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.706340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.707004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:32.707301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.709456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.723024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 216/439] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:32.729018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.733829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014f6e-d3e0-4ee9-ab80-0023e77aa740 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.753315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.779193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c45f-46f9-4a00-a5ac-e1c54ca3d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.779812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c45f-46f9-4a00-a5ac-e1c54ca3d72f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.780141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c45f-46f9-4a00-a5ac-e1c54ca3d72f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.780416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c45f-46f9-4a00-a5ac-e1c54ca3d72f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.861876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38accf-24c9-4918-83d5-8b3183733bae None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.866553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801c45f-46f9-4a00-a5ac-e1c54ca3d72f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.868100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801c45f-46f9-4a00-a5ac-e1c54ca3d72f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.905201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938b725-814f-426c-acd9-43b91b59e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.906161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938b725-814f-426c-acd9-43b91b59e577 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.906655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938b725-814f-426c-acd9-43b91b59e577 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.906928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938b725-814f-426c-acd9-43b91b59e577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.922717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.986113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 224/440] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:32.987318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caedfa-3f66-4555-ae78-bd9c554a8501 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.987601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caedfa-3f66-4555-ae78-bd9c554a8501 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.987870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caedfa-3f66-4555-ae78-bd9c554a8501 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.989922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 225/441] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:32.992930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caedfa-3f66-4555-ae78-bd9c554a8501 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.994764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938b725-814f-426c-acd9-43b91b59e577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.996293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938b725-814f-426c-acd9-43b91b59e577 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.011689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.093587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caedfa-3f66-4555-ae78-bd9c554a8501 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.093934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 217/442] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:33.095167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.095455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.095731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.113472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:33.114767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:33.132045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.132749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.133034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.133308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.134417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.135854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.137506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 226/443] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:33.138261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.152459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.188730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 218/444] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:33.189564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.231740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.250470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef16a0a-7308-449a-ab48-791442a4faf8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.287485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.288157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.288435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.288727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] PATH_INFO: `/v3/projects/c6897c5d1cf24f76b49b423cc3b01774` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.289823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] RBAC: Authorizing `identity:get_project(project_id=c6897c5d1cf24f76b49b423cc3b01774)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.291252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.293169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 227/445] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/projects/c6897c5d1cf24f76b49b423cc3b01774 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:33.298528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.299987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d388-809b-49ae-b799-15bb4dcaa5d0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.342650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.343597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.343889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.344293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.345588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.347563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.352097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 219/446] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:33.355359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.399703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.409447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d21cdc-61c3-498c-9b78-597ef6f9ad00 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.452917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.453554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.453834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.454106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] PATH_INFO: `/v3/groups/e6259141767841869cef233a128d3f51` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.455207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] RBAC: Authorizing `identity:get_group(group_id=e6259141767841869cef233a128d3f51)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.456693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.458681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 228/447] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/groups/e6259141767841869cef233a128d3f51 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:33.460218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.472223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4b2bd-29f4-47df-b4d8-2bd6ff8b1ce3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.483571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.508190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.508732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.509008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.509276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.510000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=e1f816c1dc84469786d5a59d7920b1f1, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:33.510307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.511729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.517561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 220/448] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=e1f816c1dc84469786d5a59d7920b1f1 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:33.518162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.572108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a3477-b2e0-4693-a4a0-20dc930e6f61 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.609799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.610448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.610724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.610995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.624800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:33.633972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:33.634960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:33.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:33.638101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 229/449] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:33.642338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ff649-c203-438e-abd2-c62dac0310ba None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.643791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.646437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.689096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.689704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.689980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.690253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.690980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] RBAC: Adding query filter params (group.id=e6259141767841869cef233a128d3f51, role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=c6897c5d1cf24f76b49b423cc3b01774) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:33.691327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.693536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.705112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 221/450] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/role_assignments?group.id=e6259141767841869cef233a128d3f51&scope.project.id=c6897c5d1cf24f76b49b423cc3b01774&role.id=87236176cec04669b7f8d8199ce12228 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:33.738078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.765842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1b06-bbe5-4079-ba2a-875cbe058f10 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.807705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.808375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.808817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.808919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.812831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:33.812831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.812831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.821930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 230/451] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:33.827511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.942754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5d895-a700-49b2-ae74-5ca55f1674c2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.976577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.977178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.977457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.977731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.012937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:34.050421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.052532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.148217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 222/452] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:34.154072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.255352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4568c6-6996-40ea-880b-0dc02c2718e0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.288213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:34.288778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.289061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.289336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] PATH_INFO: `/v3/projects/ad742c0425e4471ab69af90dda50deea` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.290452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] RBAC: Authorizing `identity:get_project(project_id=ad742c0425e4471ab69af90dda50deea)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.291894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.293792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 231/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/projects/ad742c0425e4471ab69af90dda50deea => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:34.299365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.363112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:34.396744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9bd0c-ff50-4001-8daf-2a9e64913528 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.430065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453a94a-65d1-4fba-9d28-3224b9d53c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:34.430666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453a94a-65d1-4fba-9d28-3224b9d53c7a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.430940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453a94a-65d1-4fba-9d28-3224b9d53c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.431209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453a94a-65d1-4fba-9d28-3224b9d53c7a None admin] PATH_INFO: `/v3/groups/c383187e76f7455fa9c6f6d63f37054a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.432307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453a94a-65d1-4fba-9d28-3224b9d53c7a None admin] RBAC: Authorizing `identity:get_group(group_id=c383187e76f7455fa9c6f6d63f37054a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.433746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453a94a-65d1-4fba-9d28-3224b9d53c7a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.435278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 223/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/groups/c383187e76f7455fa9c6f6d63f37054a => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:34.441134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.454168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:34.542937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabd65d-66a6-4c69-853f-ed528d28dbfc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.575267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:34.575858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.576223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.576494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.577202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] RBAC: Adding query filter params (group.id=c383187e76f7455fa9c6f6d63f37054a, role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=ad742c0425e4471ab69af90dda50deea) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:34.577503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.578917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa2557-9594-4b5d-8372-25cc110ee422 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.584563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 232/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/role_assignments?group.id=c383187e76f7455fa9c6f6d63f37054a&scope.project.id=ad742c0425e4471ab69af90dda50deea&role.id=580c9e29995a41c49a5a943d7b48532c => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:36.859154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d48174-624e-4dae-9442-94f448119b03 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.859939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d48174-624e-4dae-9442-94f448119b03 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.859939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d48174-624e-4dae-9442-94f448119b03 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.861454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 224/456] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:36.867754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0faf88-f540-4664-ba02-421bdbbb4b4f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.868376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0faf88-f540-4664-ba02-421bdbbb4b4f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.868376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0faf88-f540-4664-ba02-421bdbbb4b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.897122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9d0faf88-f540-4664-ba02-421bdbbb4b4f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:36.898360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9d0faf88-f540-4664-ba02-421bdbbb4b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:36.973967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 233/457] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:37.222482 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dc804-7e9a-4b20-9cc5-d990536d587e None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.222683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dc804-7e9a-4b20-9cc5-d990536d587e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.223213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83dc804-7e9a-4b20-9cc5-d990536d587e None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.225915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 225/458] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:37.231035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ed465-af5d-453c-93a0-261f93f9d32f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.231585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ed465-af5d-453c-93a0-261f93f9d32f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.232097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ed465-af5d-453c-93a0-261f93f9d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.244085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.255977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-363ed465-af5d-453c-93a0-261f93f9d32f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:37.257985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-363ed465-af5d-453c-93a0-261f93f9d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:37.346553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 234/459] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:37.416231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dc804-7e9a-4b20-9cc5-d990536d587e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.433061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6bf1f-8ef5-4ed4-9465-38869256ae0f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.433348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6bf1f-8ef5-4ed4-9465-38869256ae0f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.433621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6bf1f-8ef5-4ed4-9465-38869256ae0f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.434965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 235/460] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:37.451120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ca257-b3c9-405d-ab03-0d0e08961764 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.451403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ca257-b3c9-405d-ab03-0d0e08961764 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.451677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ca257-b3c9-405d-ab03-0d0e08961764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.481145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cd0ca257-b3c9-405d-ab03-0d0e08961764 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:37.482369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cd0ca257-b3c9-405d-ab03-0d0e08961764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:37.551447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83dc804-7e9a-4b20-9cc5-d990536d587e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.557293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 236/461] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:37.558763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e734af-cfab-4039-b340-250d7202226d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.559240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e734af-cfab-4039-b340-250d7202226d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.559702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e734af-cfab-4039-b340-250d7202226d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.565409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 237/462] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:37.567029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.568205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.568794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.587365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:37.588959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:37.624878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.626042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.628105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.628424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.641074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.652857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.654311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:37.669671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:37.670608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:37.672062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:37.672062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:37.672062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:37.672062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 226/463] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:37.672062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.683173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.697210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 238/464] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:37.698357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.848439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7345b6-b25d-40d7-bd30-0a8136c1240b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.861109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa468f-7a18-428b-937c-2a86abcc4116 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.922342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.923011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.923284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.923558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.924535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:37.924535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.926169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.928985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.929835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.930736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.931190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.940140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 239/465] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:37.941121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.946743 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.948563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:37.967473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:37.968437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:37.969294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:37.969294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:37.969294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:37.969294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 227/466] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:37.970072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.009553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.018731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.063367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef07b7-95fd-44b5-86ea-6c42fa415b4e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.086517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc18e5-6fbf-4760-9b07-de2911f2ec08 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.101883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.102548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.102837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.103048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.113624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.116085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.123102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.123699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.123993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.124270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.124901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.126347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:38.133814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:38.134684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:38.135526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:38.135526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:38.135526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:38.137360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 240/467] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:38.138570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.153298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 228/468] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:37 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:38.154266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.268965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96749a8-97af-4d7c-82f9-7d31ec5d360b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.277231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.284816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.295202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779b84-264a-48e2-92fa-94dd14e53c0a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.309951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.310586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.310871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.311151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.311811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:38.312152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.313548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.327057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 241/469] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.328064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.340595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.341678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.342183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.342674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.355235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.356844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:38.375882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:38.376943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:38.378384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:38.378384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:38.378384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:38.378384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 229/470] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:38.378384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.465358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c10f9-99ee-49cd-adae-00d6b1933b7e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.508604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb2e65-59c2-40dd-8e68-91207ebf93bd None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.513305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.514205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.514666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.515128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.515960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:38.516904 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.518932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.535902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 242/471] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.537506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.552217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.553040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.553435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.553790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.554507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.556120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.573451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.586291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 230/472] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:38 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:38.587292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.591976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.728888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d446c-e3a9-4bd6-8387-f758e7859693 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.779854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.781393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.781393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.781557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.782484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:38.782794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.784690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42518bc4-0b34-4a19-bb4b-807c241aa34d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.787360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e40f-9cd9-47da-a7fb-8f248f8d904e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.800791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 243/473] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.802578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c58504-dee0-4499-9969-70bc465f68f1 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.802797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c58504-dee0-4499-9969-70bc465f68f1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.803073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c58504-dee0-4499-9969-70bc465f68f1 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.805533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 244/474] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:38.807078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c58504-dee0-4499-9969-70bc465f68f1 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.833577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea4edf-8eaa-4424-88b1-1e85081f0f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.834563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4edf-8eaa-4424-88b1-1e85081f0f8b None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.835515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4edf-8eaa-4424-88b1-1e85081f0f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.836008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea4edf-8eaa-4424-88b1-1e85081f0f8b None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.837126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea4edf-8eaa-4424-88b1-1e85081f0f8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.839390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea4edf-8eaa-4424-88b1-1e85081f0f8b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.869257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 231/475] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:38 2022] POST /identity/v3/users => generated 248 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:38.872237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.872923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.873210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.899208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:38.900656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:38.922800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.943568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c58504-dee0-4499-9969-70bc465f68f1 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.982224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.982992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.983283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.983557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.984281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.986221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:39.014154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:39.016226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 245/476] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:38 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:26:39.038970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 232/477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:39.352009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:39.465614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58e5c-9654-4cf1-a451-0aaeb28fc303 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:39.518944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:39.519558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:39.519833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:39.520129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:39.521318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:39.522674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:39.541265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.sql.core [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:26:39.541265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 16:26:39.541265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [parameters: {'id': '688388ada70c43c4beaac7410ebf9771', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 16:26:39.541265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81687) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result.read() Aug 24 16:26:39.557805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:26:39.558694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application yield resource Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise self.value Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application session.commit() Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:26:39.560072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:26:39.561132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:26:39.562002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result.read() Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:26:39.562819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [parameters: {'id': '688388ada70c43c4beaac7410ebf9771', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:39.564316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 16:26:39.566429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:39.567872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 246/478] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 16:26:39 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:39.567972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:39.694607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8fd69-a7e4-4cde-b3d7-4b5cf32d4119 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:39.735552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc4ba0-8426-4550-9df6-8981c272756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:39.736207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4ba0-8426-4550-9df6-8981c272756f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:39.736481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4ba0-8426-4550-9df6-8981c272756f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:39.736792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4ba0-8426-4550-9df6-8981c272756f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:39.750041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc4ba0-8426-4550-9df6-8981c272756f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-16bc4ba0-8426-4550-9df6-8981c272756f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:39.755311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:39.757571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:39.759550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 233/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:39.764226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:39.780341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:39.900217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ce97a-19ff-466b-ab49-241318177935 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:39.949689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:39.951078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:39.951558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:39.952058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:39.952939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:39.953449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:39.955006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c529bfa-a53a-43a5-abea-f26c86bd8f10 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:39.967172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 247/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:40.519344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73162ba-0af2-49a3-bdea-5a16c8ca8f13 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.520902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73162ba-0af2-49a3-bdea-5a16c8ca8f13 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.521418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73162ba-0af2-49a3-bdea-5a16c8ca8f13 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.523237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 234/481] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:40.529511 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0ed99-8d00-4264-8025-a036fd199e46 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.530039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0ed99-8d00-4264-8025-a036fd199e46 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.530516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0ed99-8d00-4264-8025-a036fd199e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.549611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-10e0ed99-8d00-4264-8025-a036fd199e46 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:40.551164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-10e0ed99-8d00-4264-8025-a036fd199e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:40.622471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:40.634888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 248/482] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:41.086198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73162ba-0af2-49a3-bdea-5a16c8ca8f13 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.133446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d739-d0d3-4379-ac1a-c2285229396f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.134392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d739-d0d3-4379-ac1a-c2285229396f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.134866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d739-d0d3-4379-ac1a-c2285229396f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.137883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 249/483] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:41.142420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9fc1-8dc2-494b-95b6-574dba62bfa8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.142958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9fc1-8dc2-494b-95b6-574dba62bfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.143429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9fc1-8dc2-494b-95b6-574dba62bfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.151774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.179256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b2ac9fc1-8dc2-494b-95b6-574dba62bfa8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:41.180496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b2ac9fc1-8dc2-494b-95b6-574dba62bfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:41.242744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73162ba-0af2-49a3-bdea-5a16c8ca8f13 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.292478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.292910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 250/484] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:41.294066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2000b-ec56-4cf7-821c-73c286ba37c7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.294338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2000b-ec56-4cf7-821c-73c286ba37c7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.294614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2000b-ec56-4cf7-821c-73c286ba37c7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.296130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 251/485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:41.296874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.297226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.297510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.301237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.302343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.302825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.317111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:41.321785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:41.322400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.323270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:41.324489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:41.326191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:41.332096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 235/486] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:41.340931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.456176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 252/487] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:41.476892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65925f-52b9-4990-8171-c5be103bf5ff None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.488515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.518556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.519509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.520639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.521147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.522036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:41.522539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.524135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc325971-2c95-4ff7-ba27-58dd8e96f1ac None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.533164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 236/488] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:41.538907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.541825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfe64d-f8b1-4055-b0e8-36180fb8f1a4 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.542244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfe64d-f8b1-4055-b0e8-36180fb8f1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.542833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfe64d-f8b1-4055-b0e8-36180fb8f1a4 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.544539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 237/489] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:41.549575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.549859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.550133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.569540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:41.570850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:41.638134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.647816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 238/490] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:41.703369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc3bbc-e42a-4237-b2e2-586ff796825a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.755019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.755656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.755934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.756232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.773105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.773929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.774564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:41.790421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.791425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:41.792298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:41.792298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.792298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:41.792298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 253/491] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:41.793254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.884797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eba575-3b57-4ef2-89f7-b5441b95a8a8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.886328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.899527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89622e82-de4f-4559-9d4c-7bde7cff0759 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.923923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.924569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.924678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.925074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.925480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.937073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.937714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.937903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:41.938013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.938270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:41.943355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.944319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:41.945103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 239/492] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:41.945748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.950524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:41.956482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.957394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:41.958881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 254/493] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:41.960198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.066914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe507ea-205b-448a-8361-a0818563143f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.076515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379a365-8470-420b-b616-b8aefdea658d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.100704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.101321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.101598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.101876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.102525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.102831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.104225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.113243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.114745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.115246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.115753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.116815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 240/494] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.117457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.132101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:42.137427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.138767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 255/495] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:42.140142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.151889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.175504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.234823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80b57-91fe-4c7b-817a-3dc41a47fdf4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.249566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2017f-2ca6-4e27-9828-ba98e3d6e588 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.271548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.272272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.272552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.272826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.273492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.273863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.275167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.287847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 241/496] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.287847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.290436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.291212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.291490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.291825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.292880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.293187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.294709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.303851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 256/497] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.304540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.392820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.407491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.412449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c8798-7009-4542-b327-9f048a5e9611 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.419425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d2f3a-0eda-4bd6-8466-fdb1f529e8af None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.448320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.448956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.449237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.449702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.457228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.457362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.458584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.459326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.459423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.459730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.460820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.461134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.462497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.475793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 257/498] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.475793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:42.476054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.477020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 242/499] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:42.478604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.581998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9660fd3-b121-4bf4-81b9-b1e96f58815a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.584385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcff85-2675-4616-865c-0fef2d301726 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.616092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.616716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.617169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.617447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.619311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.620300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.620783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.621184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.627758 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.629214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.637330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.639545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.642347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:42.647750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.648810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:42.649649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:42.649649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:42.649649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:42.649649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 258/500] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:42.650482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:42.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.660321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:42.661161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:42.661161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:42.661161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:42.661161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 243/501] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:42.661370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.666940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.756912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69338b6-65a2-475d-af7d-d8ddc18092c8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.763366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea6dac-af4a-4505-a354-50ee72e0421c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.789175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.789813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.790083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.790357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.798333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.798936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.799217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.799496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.800295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.800602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.802311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.802841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.803864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b912a44b-b7b9-41a7-9dda-27cc9e992970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:42.818440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.819315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:42.820195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:42.820195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:42.820195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:42.820195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 259/502] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:42.820803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52a04-8545-4fdb-bb02-25300d99e8c7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.820976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52a04-8545-4fdb-bb02-25300d99e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.821248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52a04-8545-4fdb-bb02-25300d99e8c7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.822460 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 260/503] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:42.823143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52a04-8545-4fdb-bb02-25300d99e8c7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.843618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 244/504] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.844487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.881897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.906723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.925554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52a04-8545-4fdb-bb02-25300d99e8c7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.951602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5170a-defb-4ce5-917b-0d3fa46edc4f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.958978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.959591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.959873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.960264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.960911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.961215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.962591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.975735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 261/505] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.976649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.990627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.991232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.991509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.991779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.992458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.992762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.994140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a65fb7-67de-4e4b-8924-a3002d6104fe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.005969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 245/506] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:43.007188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.007453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.007721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.024818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:43.025993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:43.081755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9731b-ff63-4a32-ac57-dbca4b01c720 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.094882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 246/507] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:43.095859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.099881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.118506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.119120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.119406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.119686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.120365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:43.120673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.122058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.133700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 262/508] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:43.134504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.165109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.222977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3414e-f34f-436b-a31b-e0d4df4c023d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.242740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e22d10-8904-4904-9b6a-3d05998aec1e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.259383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.260079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.260363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.260692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.279262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.279885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.280190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.280462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.288731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.290166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:43.303355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:43.304550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:43.305437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:43.309223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 263/509] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:43.309815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.336639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.338075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.351492 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.385268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.414904 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb7efb-284b-4d99-9ab2-d1b1a26b77b7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.440540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 247/510] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:43.441293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.448040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.448667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.448947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.449216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] PATH_INFO: `/v3/users/glance` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.456988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.459402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:43.470206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:43.471233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:43.472127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 264/511] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:43.472672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.545635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa9448-b25c-47c8-8353-3f0a3826f99a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.569458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.573488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8f47e-1332-4dfc-93c5-20dc36165dba None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.580144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.580770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.581045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.581315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.589160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.590615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:43.602689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:43.603767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 248/512] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:43.605287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.611389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.611769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.612017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.612299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.612572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.613538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:43.613881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.615323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.653811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 265/513] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:43.654749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.709370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaf7ae-b3ee-414a-bc5f-78d5bfdf7183 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.742559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.743211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.743488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.743757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.753792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ec5fb-8b87-4643-81b0-24a54c61e369 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.754084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.757185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:43.771636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:43.772631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:43.773476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:43.773476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:43.773476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:43.773476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 249/514] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:43.773476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.790461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.791089 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.791365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.791635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.802405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.803861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:43.818302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:43.819182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:43.820071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:43.820071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:43.820071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:43.820071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 266/515] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:43.820279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.830482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.836906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:43.881672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46574857-f9ae-47fc-87e1-b3fd2fde631e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.915196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.915859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.916167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.916445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.917178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:43.917726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.919173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.920384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d11bb-4ec3-4bda-8d8d-72a032647d0f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:43.938556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 250/516] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:43.939253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:43.957626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:43.958369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:43.958629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:43.958904 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:43.959688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:43.959970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:43.961398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:43.978733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 267/517] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:43.979717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.043759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32893da1-e2f8-439e-9f63-7d7a0c43bab7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.053269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.058303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.079898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.080532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.080807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.081077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.089065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fd831-a554-490c-b8ae-a77063a20b70 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.123712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.124345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.124622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.124899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.126022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:44.126325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.127708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.138958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 268/518] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:44.139741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.154029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.155514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.238998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb7daa-4c7f-4aac-91b3-5708487aab73 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.254059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 251/519] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:44.255072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.264359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.271242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.275700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.276698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.277233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.277719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.278690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:44.278994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.280465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.294199 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 269/520] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:44.295014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.359262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5a0e8-b651-436f-a8ce-2024718bca32 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.393414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.394021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.394303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.394580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.397863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54d3c-fcc9-472b-954c-e931962ac80c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.430685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.431278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.431554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.431821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.461768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.463220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.477508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.487385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.501587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.502960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.564346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 252/521] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:44.565361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.601699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 270/522] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:44.602547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.667345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f371d-0a75-4522-a419-fcaebed7926b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.681203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.686806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.702275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84279eea-b8cb-44d4-bd9c-79eb0d4fcf75 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.703178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.703791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.704115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.704395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.705045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.706508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.728742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 253/523] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:44 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:44.729550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.737403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.738051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.738326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.738593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.752070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.753663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:44.771898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:44.772909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:44.773761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:44.773761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:44.773761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:44.773761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 271/524] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:44.773976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.865682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e831-5453-469f-9ffc-990223e1dd02 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.893286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434233-ca26-4840-9808-cf9ce1146380 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.904019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.904601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.904878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.905145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.912823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.914808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:44.930461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:44.931457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 254/525] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:44.933021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.938116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.938827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.939172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.939506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.948280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.953381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:44.955053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:44.970303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:44.971249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:44.972201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:44.972201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:44.972201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:44.972363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.972438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 272/526] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:44.972438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.058608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e583-d8c3-41f5-b52f-e99f00d7399b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.085283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702f22a-213e-465b-9434-4533b5b5b9c0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.098761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.099464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.099747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.100051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.111235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.112727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.124121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.124966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.125479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.125930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.126796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.127301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:45.127453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.128356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:45.129286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:45.129286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:45.129286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:45.129286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 255/527] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:45.129286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.132586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.145107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 273/528] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.146271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.200493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.226147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.246526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5350ed5-5793-4542-92f5-6c590f6449ab None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.255518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b1a33-f933-4aa4-a79f-ba6f0fc1e89a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.281368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.281975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.282250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.282572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.283215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.283516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.284973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.291720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.293112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.293463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.293802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.294516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.295068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.296760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.306102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 256/529] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.306977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.321307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 274/530] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.321307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.418164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1eb1ae-67fe-4cc3-806b-6dea3d512cf2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.425962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075cf07-5459-4724-a371-05ca1d539d51 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.437610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.455376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.456176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.456455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.456727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.457472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.457935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.459313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.461920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.462608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.462953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.463292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.470113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.475711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 257/531] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.475711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.475917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.478155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:45.488557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.489830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:45.490726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:45.494566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 275/532] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:45.495624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.585139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20854197-75ab-410a-921f-54e9967bb493 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.603154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b7666-e51b-43eb-bcc2-1ff06315032e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.619180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.619785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.620095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.620385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.621109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=f1ab09f1996244a2b328a887fe91febd) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.621417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.622897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.628324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 258/533] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/role_assignments?user.id=f1ab09f1996244a2b328a887fe91febd&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.629228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.650314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.651418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.651875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.652376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.662065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.664487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:45.680493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.681406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:45.682224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:45.686496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 276/534] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:45.687582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.723265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.730835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.788685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b0cdf-9248-42f4-b96f-a4038b994208 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.809097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f00c7b-80bd-4eaa-a54e-54d6687b8c9b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.828811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.829491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.829774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.830047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.840221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.842298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.846221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.847039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.847455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.847866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.848824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.849373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.851229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:45.856770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.857683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:45.858540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 259/535] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:45.858993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.877882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 277/536] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.878813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.989563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b493-19d7-4409-9b3b-8d8fed7aaab9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.001945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.011137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a201317-b8a2-4d4f-988d-9888fe769725 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.024967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.041211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.041838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.042199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.042803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.043656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.044059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.045653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.058284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.059152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.059602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.064504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 260/537] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.064504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.064705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.065276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.065276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.066671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.083988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 278/538] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.083988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.197558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef0edc-ce4c-402b-9200-2590890c3a11 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.198135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc9013-5442-4458-9b61-652a1b261089 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.235884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.236790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.237204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.237348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.237739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.238074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.238445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.239011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.239452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=11df13be0c5543748f98b9ea3d83c3b8) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.239976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.240098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=3671de0f76cc475db615dcc9b351c271) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.240490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.241812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.242097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881be4d-defb-4245-933b-ea50c0447a8c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.249131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 261/539] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/role_assignments?user.id=11df13be0c5543748f98b9ea3d83c3b8&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.250176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.256623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 279/540] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/role_assignments?user.id=3671de0f76cc475db615dcc9b351c271&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.319046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.380558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492b9e0-b89a-48ea-b13f-13a48ef52aa7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.421440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.422338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.422756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.423163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.424069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=505ea6d7aac346e1a604b192a2a0c364) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.424576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.426185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e8eb9-2a3b-4537-8066-1ec86694ca2f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.433715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 262/541] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/role_assignments?user.id=505ea6d7aac346e1a604b192a2a0c364&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.737543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87e81e-e67a-4d50-acbe-17c5e349ad52 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.738078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87e81e-e67a-4d50-acbe-17c5e349ad52 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.738078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87e81e-e67a-4d50-acbe-17c5e349ad52 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.740332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 280/542] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:46.746179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919731a0-564d-400e-a487-309a2d74451b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.747052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919731a0-564d-400e-a487-309a2d74451b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.747522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919731a0-564d-400e-a487-309a2d74451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.772149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-919731a0-564d-400e-a487-309a2d74451b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:46.773724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-919731a0-564d-400e-a487-309a2d74451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:46.864953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 263/543] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:47.201603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87e81e-e67a-4d50-acbe-17c5e349ad52 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:47.220815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:47.329430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87e81e-e67a-4d50-acbe-17c5e349ad52 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:47.374387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:47.375023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.375303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.375581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:47.387082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:47.391807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:47.392728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:47.393505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 281/544] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:47.399038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919731a0-564d-400e-a487-309a2d74451b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:47.470498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:47.536678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919731a0-564d-400e-a487-309a2d74451b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:47.579323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:47.580357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.580837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.581315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:47.582181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:47.582758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:47.584346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:47.593142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 264/545] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:47.601020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:47.691434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:47.718468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a070-2347-4bf2-90cc-ee33152d0ec5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:47.752835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:47.753577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.753854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.754128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:47.765629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:47.767121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2885b4a3-48b3-41c7-bf5e-217ce399ce55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:47.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:47.784720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:47.785534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:47.785534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:47.785534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:47.785534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 282/546] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:47.792870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:47.838377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33711411-1334-48d6-a9f9-45b78bca9c49 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.838377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33711411-1334-48d6-a9f9-45b78bca9c49 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.838606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33711411-1334-48d6-a9f9-45b78bca9c49 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:47.840109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 283/547] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:47.845014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.845295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.845569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:47.864717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:47.866048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:47.930236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6e1c-a173-4ec1-af99-d7244d92e6f9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:47.943164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:47.948943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 284/548] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:47.994793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:47.995427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.995684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.995974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.001883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:48.002199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.003683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.026013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 265/549] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:48.031731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.112281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:48.162370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d409-b5ac-4b26-a65b-be221bbc9618 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.203366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.204019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.204302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.204576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.212359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.213792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:48.232815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:48.234230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:48.235293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 285/550] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:48.241026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.264277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.387813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdcf58-ceb9-48a5-9be3-4075e5a90149 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.409654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:48.430338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.431349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.431837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.432350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.444023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:48.449428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:48.450283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:48.451102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:48.453819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 286/551] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:48.460306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.509924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1811737-6988-49fa-8092-859511a1d4b4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.573243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b06db9-2181-42ac-bdc7-a2063896c3ae None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.602306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.602927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.603205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.603477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.604248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:48.604555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.605986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.637165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 266/552] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:48.643298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.643857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.644175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.644456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.645713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:48.646014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.647380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.649526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.666972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 287/553] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:48.685730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.700636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:48.760690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:48.818709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861b75b-2baf-4aea-91f0-2c554d9aff24 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.835131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d666ec-24d4-4cf1-986a-60f1a9d6f335 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.858517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed115bf5-6da7-4e13-9421-a2bfc999adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.859846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed115bf5-6da7-4e13-9421-a2bfc999adc1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.859846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed115bf5-6da7-4e13-9421-a2bfc999adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.860202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed115bf5-6da7-4e13-9421-a2bfc999adc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.873786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.874425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.874701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.874969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.886896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.888445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-1de74ecc-829e-4531-84c4-8da8dd2790fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:48.903913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:48.904978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:48.905980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:48.905980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:48.905980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:48.905980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 288/554] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:48.906256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb616153-e827-4492-8586-da0240c85afb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.906429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb616153-e827-4492-8586-da0240c85afb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.906758 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb616153-e827-4492-8586-da0240c85afb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.907879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 289/555] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:48.914227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc1bc4-143c-4403-8fc1-8aa62aa1483e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.914513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc1bc4-143c-4403-8fc1-8aa62aa1483e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.914789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc1bc4-143c-4403-8fc1-8aa62aa1483e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.919976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 290/556] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:48.919976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc1bc4-143c-4403-8fc1-8aa62aa1483e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.945242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed115bf5-6da7-4e13-9421-a2bfc999adc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.946696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed115bf5-6da7-4e13-9421-a2bfc999adc1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.972833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.019078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.031396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc1bc4-143c-4403-8fc1-8aa62aa1483e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.060001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 267/557] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:49.060316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b48a6b-d33b-4f57-b16f-6334398fd74f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.060605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b48a6b-d33b-4f57-b16f-6334398fd74f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.060885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b48a6b-d33b-4f57-b16f-6334398fd74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.067888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.068521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.068801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.069081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.069749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:49.070055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.071421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3755e53-05e3-4d10-a97b-09db36729966 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.078744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-66b48a6b-d33b-4f57-b16f-6334398fd74f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:49.079964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-66b48a6b-d33b-4f57-b16f-6334398fd74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:49.083773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 291/558] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:49.084996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.085278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.085556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.103505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:49.104736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:49.147277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 268/559] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:49.148489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dc9c0-9292-4d5a-b68d-d658d6c08de5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.148771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dc9c0-9292-4d5a-b68d-d658d6c08de5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.149267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dc9c0-9292-4d5a-b68d-d658d6c08de5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.150701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 269/560] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:49.151399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dc9c0-9292-4d5a-b68d-d658d6c08de5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.175906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 292/561] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:49.177548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.228959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.279304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dc9c0-9292-4d5a-b68d-d658d6c08de5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.295117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.324221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a8f16-400a-4cef-b5bb-f9f009c06bfb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.325671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.326305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.326590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.326863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.340058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.341793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6be7a6f7-8735-40c8-8cda-b113435920bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:49.361894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:49.362873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:49.364287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:49.364287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:49.364287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:49.364287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 270/562] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:49.365167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.365456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.365762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.367984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.369007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.369546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.370115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.378647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.381289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.387169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:49.388450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:49.395350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:49.396362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:49.397262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:49.400031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 293/563] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:49.401014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.471087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 271/564] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:49.471844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.508265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.511751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07206-8274-4126-8a00-507b897d3dec None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.548667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.548981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.549235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.549512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.550150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:49.550452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.552956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.563513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.572503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 294/565] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:49.574052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.585329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74246ac-b0a6-41f9-9341-e91709eb28b3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.621563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.622172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.622453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.622734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.623467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:49.623775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.625417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.648114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 272/566] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:49.648114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.694262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5878d7-4915-45b2-95f9-80a79e2cdf7a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.733054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.733615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.733893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.734166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.745915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:49.746152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:49.750649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:49.751581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:49.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 295/567] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:49.753153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.756725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6e89f-cfb3-465d-8ca2-c8f530b3a0eb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.790541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.791162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.791434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.791702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.803218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:49.807911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:49.808867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:49.809690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 273/568] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:49.810749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.816238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:49.858425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb092f0-66df-4b51-b153-5116a0940198 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.892603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.893227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.893508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.893784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.900806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.902241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.909716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b976c-5278-4499-992d-1ddaa65ae066 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:49.914119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:49.915156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 296/569] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:49.916082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.948255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:49.948877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:49.949157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:49.949433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:49.959915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:49.961465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:49.976511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:49.977631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:49.978662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:49.978662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:49.978662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:49.978662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 274/570] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:49.978662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:49.986868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.021417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8c47a-4a9e-4fd1-9df1-a0c4b7e1d251 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.042184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.056540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.057163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.057439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.057707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.067796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:50.072839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:50.073931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 297/571] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:50.074962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.088528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9c3dc-3e94-445b-b13a-77e09bf540c4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.121968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.122574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.122855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.123131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.123798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.124137 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.125523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.133983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 275/572] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.135050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.174632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57e54c-a7ac-42f7-97e5-76aebd0982c8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.203770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.208212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.208892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.209172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.209445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.210097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.210395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.211743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.219552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 298/573] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.220402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.243218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61912-809a-46c0-8fb1-573a29958074 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.264836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.278775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.279377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.279653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.279923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.280655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.280935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.282349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.294675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 276/574] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.295632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.325251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a737e95-2fab-4066-ac1d-ed781aee7856 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.359538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.360166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.360445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.360718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.361352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.361650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.363000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.375065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 299/575] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.375966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.397349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ac78-7254-47d7-878f-4a553ea5c218 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.424790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.433403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.434008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.434279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.434547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.435232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.435529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.436920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.444948 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 277/576] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.446283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.478940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda870d-45fd-4012-b166-2ceae30739b9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.480530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.514681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.515240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.515514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.515792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.525841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.527278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:50.543193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:50.545311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:50.546419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:50.546419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:50.546419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:50.546419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 300/577] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:50.546419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.550080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5806448-8457-4705-acfd-c9b42af2ad49 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.585620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.586246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.586519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.586791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.597772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.600014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:50.614611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:50.615574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:50.616650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:50.616650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:50.616650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:50.616650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 278/578] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:50.616650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.644821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe1130-44e8-4348-8b09-f3a7902b9588 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.661995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.679465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.680097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.680377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.680657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.681377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.681686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.683099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.688856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 301/579] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/role_assignments?user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.689632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.712173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.733411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87660cf8-2014-4cc9-8bdc-b9b8fca4b44d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.774779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.775338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.775622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.775898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.783500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.785023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:50.798192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:50.799305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 279/580] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:50.800699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.814054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eeb7d-7fe4-4eda-abec-eced18bc786e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.850909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.851542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.851931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.852313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.864470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.867309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:50.885054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:50.886125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:50.887018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:50.887018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:50.887018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:50.887018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 302/581] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:50.887604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:50.918878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec97fcd-4948-496c-b4ff-c690c638600e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:50.920442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.957603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:50.958272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.958542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.959043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.959459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:50.959767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:50.961174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:50.966961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:50.978939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 280/582] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:50.978939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.010567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd6864-ea31-4ff6-a5a7-2319f48e0ee5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.052589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.053460 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.053889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.054336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.055132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:51.055594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.057011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.071668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 303/583] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.071668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.094296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430ec0a-7d47-4ecb-a6fc-51412e4a3b0e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.130095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.130821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.131107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.131385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.132117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:51.132427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.133861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.145313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 281/584] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.146103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.172804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.193607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8c4e-8bb2-4894-a7e8-05c46ca9020f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.228635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.235172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.235776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.236104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.236372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.237012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:51.237316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.238714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.250646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 304/585] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.251341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.259925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bef00-ac05-40e5-b933-772ebab56e34 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.298534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.299511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.299981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.300510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.308904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.310795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:51.323282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:51.324468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:26:51.325641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:51.328981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 282/586] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:51.329918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.363719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa3b2-b628-4759-9ee8-1ae3e9c49019 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.398611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.399241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.399521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.399851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.407238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.408724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:51.420811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:51.421805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 305/587] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:51.422766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.425097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.444945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7702-9b23-4153-8cb3-2becef3afda9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.480561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.480869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.481531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.481811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.482087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/f1ab09f1996244a2b328a887fe91febd/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.497893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=f1ab09f1996244a2b328a887fe91febd, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.499449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.524071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 283/588] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:51 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/f1ab09f1996244a2b328a887fe91febd/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:51.524071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.553079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac17128-40d4-498f-9d5d-0b8c1eb2885f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.595474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.596334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.596780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.597209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.607217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.609024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:51.627576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:51.628632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:51.630242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 306/589] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:51.635110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.658466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c64908-a148-4aa1-ade0-560c3f11e845 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.696820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.703338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.703997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.705625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.705974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.707267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:51.707267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.709238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.735337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 284/590] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.736349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.762070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4d4f-b05b-4219-9920-072e47236fb8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.775124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.802867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.803540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.803771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.804082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.804924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:51.805236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.806829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.826644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 307/591] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.827500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.852494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41bd87-3cf6-424e-8d17-6d6377acb9b5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.889617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.890357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.890640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.890915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.891649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:51.892083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.893664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.916942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 285/592] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.918049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.944578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67ab8b-9b4d-4ca9-a50b-bf4a889746c8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.981623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.984154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.984807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.985093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.985390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.001792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.003393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:52.027105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:52.028493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:52.029361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:52.029361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:52.029361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:52.031442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 308/593] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:52.032603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.042723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb153e47-cd33-4bf0-a07f-d710269ecb46 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.060372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.083965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.084601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.084876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.085223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.095625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.097400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:52.113031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:52.114310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:52.115820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:52.115820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:52.115820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:52.115820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 286/594] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:52.115820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.147377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8159f-4af1-4b55-be4b-80272337acec None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.183167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.184251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.184745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.185212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.197674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.199422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:52.214702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:52.215666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:52.216599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:52.216599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:52.216599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:52.218946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 309/595] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:52.220113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.225795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.228695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7902f2-e74a-4cb2-96db-aa8827eabeeb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.266332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.266944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.267226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.267501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.268284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.268588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.270577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.283779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 287/596] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:52.284786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.299619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.332172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594804b9-7412-449a-a593-aa2937a002e9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.375933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.376722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.377095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.377466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.378321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.378805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.381507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.400007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 310/597] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:52.400751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7eb886-2a14-4d99-a804-f45bcff4eac3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.401334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.439886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.440865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.441381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.442497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.443343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.443852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.445435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.458533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 288/598] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:52.459806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.483028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.512700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427552-c65d-4852-8acb-3329f65ef8e8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.544975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.550452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.551973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.552477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.553674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.562748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.564659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.573719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54c159-c86a-44b3-92a0-ba5df7ce59ad None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:52.577136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:52.577972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:52.578859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:52.581458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 311/599] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:52.583096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.609273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.609908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.610186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.610462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.618297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.620128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:52.632266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:52.633191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:52.634092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 289/600] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:52.634578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.698169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d5757-e81f-4ddc-8046-733e348ecb34 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.728635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.737197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.737623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.737901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.738172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.745880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.747346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.750754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb941ab-8b7b-48fe-ad2b-acaf2c13e0b6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:52.761379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:52.762339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 312/601] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:52.767977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.795335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.795936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.796252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.796516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.797199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.797503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.798910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b27257-0d7b-417d-a972-1e7bfc1a2c36 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.811900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 290/602] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:52.817797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1be12-e1e6-4533-8446-e526ef97f5df None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.818099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1be12-e1e6-4533-8446-e526ef97f5df None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.818353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1be12-e1e6-4533-8446-e526ef97f5df None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.823576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 291/603] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:52.824333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1be12-e1e6-4533-8446-e526ef97f5df None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.824870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:52.874511 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ea3b-26ed-4650-a62f-ac69a29b2df3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.912837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.913452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.913734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.914015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.914767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.915095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.916568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.931175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 313/604] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:52.932058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.942634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1be12-e1e6-4533-8446-e526ef97f5df None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.978804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.979407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.979687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.979977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.980682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.980985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.982406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c5b47-02fa-4bbe-8e72-f4bc9f3837ce None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.992048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:53.000045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 292/605] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:53.000045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.000045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.000045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.018355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:53.019701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:53.054392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1ce02-d802-4056-b955-93f5cce78990 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:53.067461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:53.089361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:53.090031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.090308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.090596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/3671de0f76cc475db615dcc9b351c271/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.104041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 293/606] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:53.104041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:53.107161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=3671de0f76cc475db615dcc9b351c271, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:53.108736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:53.123331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 314/607] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:52 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/3671de0f76cc475db615dcc9b351c271/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:53.124634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:53.218013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b577e9-c006-4015-928d-1a0e1c93da03 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:53.249041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3be6-5c7c-4ec7-8051-64eedf7211eb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:53.291957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:53.292913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.293729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.294021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/505ea6d7aac346e1a604b192a2a0c364/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.306866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:53.315727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=505ea6d7aac346e1a604b192a2a0c364, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:53.317879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:53.324709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9b207-b6cc-4e82-ba44-8e7794b4986f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:53.325718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9b207-b6cc-4e82-ba44-8e7794b4986f None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.325841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9b207-b6cc-4e82-ba44-8e7794b4986f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.326069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9b207-b6cc-4e82-ba44-8e7794b4986f None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/11df13be0c5543748f98b9ea3d83c3b8/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.334541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 315/608] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:53 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/505ea6d7aac346e1a604b192a2a0c364/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:53.346757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9b207-b6cc-4e82-ba44-8e7794b4986f None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=11df13be0c5543748f98b9ea3d83c3b8, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:53.348400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9b207-b6cc-4e82-ba44-8e7794b4986f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:53.368307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 294/609] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:53 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/11df13be0c5543748f98b9ea3d83c3b8/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:53.511918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:53.530754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:53.618599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364095-73ac-47d3-82ec-bcaa49f13e77 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.618907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364095-73ac-47d3-82ec-bcaa49f13e77 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.619161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364095-73ac-47d3-82ec-bcaa49f13e77 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.620563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 295/610] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:53.627276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.627559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.627822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.644706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd077f-a07c-4144-b0d2-e6068be03e45 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:53.650484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:53.651687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:53.718410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:53.718410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.718410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.718410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.740506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:53.746111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:53.747049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:53.748101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 316/611] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:53.753040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:53.777411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 296/612] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:26:53.838215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:53.871449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5929-e2ac-434b-8db6-39396a304f82 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:53.911423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:53.912404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:53.912898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:53.913384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:53.914293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:53.914798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:53.916376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:53.925661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 317/613] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:53.931564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.040933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd590-2f15-4d7b-92d9-ff2f789d8a61 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.060333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.078246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.078873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.079147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.079418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.090135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.091581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:54.114728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:54.115685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:54.116586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:54.116586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:54.116586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:54.120331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 297/614] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:54.121447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.199305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.237112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc54b-7a85-4e8d-ac65-dd02d1abefda None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.306786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.307314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.307597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.307874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.308544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:54.308849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.310217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.317980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.334633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 318/615] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:54.344890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.390626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37aac-72aa-48ee-8f7c-8154ad9d896c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.445012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.446102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.446385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.446663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.459275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:54.464847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:54.465758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:54.466567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 298/616] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:54.473973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.517416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.541273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9ee6-2871-478a-b3c3-f095b180ff0e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.600778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.601329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.601602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.601869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.618276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.619721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:54.643145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:54.644180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:54.645144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 319/617] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:54.651651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.657253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3d873-5a6a-4734-9a11-f61e23a947bd None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.731756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.732503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.733405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.733405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.733723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:54.734030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.737520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.740766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.751152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 299/618] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:54.756919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.791484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d29fd-8a70-46e5-aeb9-d6cc0c8c50a7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.827815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.828803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.829280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.829746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.830687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:54.831288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.833835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.852975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 320/619] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:54.863510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.897048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.934730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf645ac-8035-407b-a67a-2e81d68ebe9d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.985603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efee56-63ed-46b2-9572-fcc854cd5843 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.008700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.009679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.010153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.010641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.016287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.025721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.027516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:55.045880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:55.047393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:55.048301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:55.048301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:55.048301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:55.050365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 300/620] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:55.056200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.071242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.072348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.072868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.073376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.105325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.106730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:55.135373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:55.137163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:55.138247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:55.138247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:55.138247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:55.138247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 321/621] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:55.142184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.245525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d46c-2f48-41a4-9204-f44452b0612f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.299754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b7c5-a45d-4bcb-87a7-58b76c2123d2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.336928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.337900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.338373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.338862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.339689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:55.340208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.341263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.341898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.342177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.342449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.342754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.343095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:55.343394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.344907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.353070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.358775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 322/622] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:55.365750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.373019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 301/623] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:55.378315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.400309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.527343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf76b5-34de-421e-879d-6c38d0747885 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.571607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec8a3-b311-493a-86c6-fc453f6b2e46 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.590165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.591049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.591422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.591792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.599659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.601466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.610208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.611052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.611480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.611902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c68fb59e-82b4-4e5a-b29f-96c28db222ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:55.614056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:55.615119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:55.616252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:55.622451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 323/624] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:55.622451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10618b06-1bc7-49c7-a9fa-e85b147af0c5 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.622451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10618b06-1bc7-49c7-a9fa-e85b147af0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.622451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10618b06-1bc7-49c7-a9fa-e85b147af0c5 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.622451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 324/625] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:55.623176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18b787-bbc8-489d-b37e-ff5cb475a3fb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.623541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18b787-bbc8-489d-b37e-ff5cb475a3fb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.623906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18b787-bbc8-489d-b37e-ff5cb475a3fb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.625441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 325/626] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:55.626287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18b787-bbc8-489d-b37e-ff5cb475a3fb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.628443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.630517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b82a2ff2-6eb2-4cb4-87ab-a9e306342c50 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:55.647395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:55.648619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:55.649790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:55.652618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 302/627] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:55.654416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca2fa-bd94-4484-ac9d-65e6ca760dec None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.655114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca2fa-bd94-4484-ac9d-65e6ca760dec None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.655527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ca2fa-bd94-4484-ac9d-65e6ca760dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.674787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e47ca2fa-bd94-4484-ac9d-65e6ca760dec None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:55.676806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e47ca2fa-bd94-4484-ac9d-65e6ca760dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:55.719807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.731997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.740920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18b787-bbc8-489d-b37e-ff5cb475a3fb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.750402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 303/628] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:55.751634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.751917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.752222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.772343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:55.773552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:55.785257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.786407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.786679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.786949 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.787638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:55.787939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.789544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.808649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 326/629] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:55.809824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.846899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 304/630] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:55.847894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.925472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096032c-7fae-400a-90b8-49a562efdd5a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.956562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8493-65b0-4d32-bffc-445dd4502250 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:55.964270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.964967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.965246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.965517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.966236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:55.966535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:55.968083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eabe3-cb22-4b83-98d1-b14f7b66c007 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.974574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.987178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 327/631] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:55.989834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094cd07-40b4-4cbb-bea7-1d5352b0bb99 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.990111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094cd07-40b4-4cbb-bea7-1d5352b0bb99 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:55.990475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094cd07-40b4-4cbb-bea7-1d5352b0bb99 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:55.992052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 328/632] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:55.994101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094cd07-40b4-4cbb-bea7-1d5352b0bb99 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:55.996686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1c866-2bf0-46a6-84ee-5bbd399a2ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:55.997594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:55.998657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c866-2bf0-46a6-84ee-5bbd399a2ebf None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:55.999814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c866-2bf0-46a6-84ee-5bbd399a2ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.000119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c866-2bf0-46a6-84ee-5bbd399a2ebf None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/d1e90301e20d4d4ea7f11c7bef1ef4c8/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.020271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1c866-2bf0-46a6-84ee-5bbd399a2ebf None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=d1e90301e20d4d4ea7f11c7bef1ef4c8, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:56.022346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1c866-2bf0-46a6-84ee-5bbd399a2ebf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:56.048162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 305/633] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:55 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/d1e90301e20d4d4ea7f11c7bef1ef4c8/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:56.052262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.052582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.055972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.075056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:56.076462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:56.126415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094cd07-40b4-4cbb-bea7-1d5352b0bb99 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.160002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 306/634] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:56.160983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.171195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.171761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.172382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.260227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:56.262274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:56.266523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:56.280899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:56.291075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2feed-33ba-4a67-a3a9-096398f805df None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.340467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.341673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.342465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.342945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.356814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:56.362883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:56.364146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:56.365195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:56.368050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 307/635] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:56.369440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.403875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 329/636] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:26:56.405254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.496648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97ff1-9747-4ffc-90eb-8d50b379d5b4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.519481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c5b0e-4175-4e6b-8077-6db8d82d9d59 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.521626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:56.540674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.541312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.541593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.541866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.552800 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:56.558308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:56.559219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:56.560149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:56.560785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.560883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 308/637] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:56.560961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.560961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.560961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.560961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:56.561743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:56.562594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:56.564977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.572907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 330/638] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:56.574995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.585051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:56.676457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031780-3f7a-460b-89f6-8e88329d5922 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.685003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6301e12-e505-423e-b6e1-ab3ebf016025 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.712640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.713381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.713768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.714053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.721929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.722560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.723402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.723683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.725922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:56.727390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:56.735361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:56.741739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:56.742698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:56.743538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:56.744883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:56.745741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:56.746621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:56.746621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:56.746621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:56.746797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 331/639] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:56.746797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.752595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 309/640] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:56.752595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.776666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:56.820155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:56.855970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002eeab8-5c79-4db8-8c61-0667d4652c5e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.863061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc974509-9304-4e59-806a-3c120f34e348 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:56.897868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.898502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.898784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.898886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:56.899067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.899608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.900006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.900293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.900957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:56.901263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:56.902634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:56.911268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:56.915995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 332/641] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:56.915995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:56.916250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:56.934805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:56.935852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:56.943460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:56.943460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:56.943460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:56.943460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 310/642] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:56.943460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.012865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.031871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8bb3-ea4c-4485-bdbe-515907dffb21 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.056100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a507d39-7583-4e69-b481-2717b8b153fe None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.077429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.078386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.079150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.079323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.079587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.080399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.080731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.082098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.095056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 333/643] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.095056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.096988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.097611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.097937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.098197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.098907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.099225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.100615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.118505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 311/644] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.118505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.207971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34aa74-c199-49e9-ab45-795e9414ae7c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.229058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8990b3-a998-4614-b493-bba803de1ee1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.243764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.244430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.244710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.244992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.256014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.257456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.265272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.266572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.267177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.267442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.267687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.268354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.268650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.270016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:57.272816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:57.273945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:57.275155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:57.275155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:57.275155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:57.286296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 312/645] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.286465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 334/646] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:57.286568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.286663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.328280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.389514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495ec38-ee71-46e7-bbd6-106c3f7de0b1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.394739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05364cf8-7712-461d-b178-fe8df024e5e2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.424069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.424701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.424977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.425304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.430136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.430770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.431050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.431326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.436683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.438059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.438172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.439474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:57.451551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:57.452705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:57.453587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:57.454029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:57.454891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:57.455782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:57.455782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:57.455782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:57.457781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 335/647] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:57.458269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 313/648] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:57.459003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.459170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.535050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.559425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.569251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3d8c4-ddd1-4765-bfda-23957e7e27e0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.579986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef52ff6-0cea-40f5-acb2-8bef4b6a4143 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.605770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.606513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.606789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.607174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.614850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.615211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.615889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.616259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.616375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.616676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.617338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.617706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.619120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:57.627842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:57.629025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:26:57.630308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:57.632196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 314/649] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:57.633283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.637244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 336/650] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.638109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.744284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421779e-501f-40ed-9b0b-5dece25d3255 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.760356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7c885-ea64-421c-9a4d-21448665ece4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.766689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.783067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.783685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.783968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.784265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.784908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.785211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.787376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.798779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.799389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.799668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.800022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.800720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.801067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.806345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 337/651] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.806345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.806568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.821132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 315/652] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.821132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.831567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.924215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46113a89-cc4d-40f7-9c22-8f3a238d8dcf None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.924700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe626f-806e-40e4-aa14-9a57416c86fc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.961067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.961677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.961956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.962057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.962255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.962683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.962959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.963054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:57.963229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.963340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.964729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.970775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.972223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.982503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 338/653] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.983328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:57.985404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:57.986298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:26:57.987275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:57.993936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 316/654] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:57.993936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.027922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.070080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.089760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988f092-d09b-4fab-98ca-9d6f654e2e74 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.101160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f82d5-ef85-4f82-8d87-e69bce78db76 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.122724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.123347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.123624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.123895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.131777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.133252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.139818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.140465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.140747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.141021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.141752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=f1ab09f1996244a2b328a887fe91febd) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.142061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.143507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.150425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 317/655] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/role_assignments?user.id=f1ab09f1996244a2b328a887fe91febd&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:58.151237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:58.152337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:58.153287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:58.154210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:58.156951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 339/656] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:58.157110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.270125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44060c8-6178-42d5-87d3-f319cc4ec1de None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.274836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.280893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed773-bdfb-454e-8c43-a83b6b5dee49 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.308132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.308756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.309039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.309796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.316654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.325598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.326733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.328208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.328208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.328208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.328485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.330281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075455-ef03-498d-9fe1-48c2176564e5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.351369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 340/657] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:58.352573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abe263-5904-494b-ad6c-a030486e1ba1 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.352855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abe263-5904-494b-ad6c-a030486e1ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.353133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abe263-5904-494b-ad6c-a030486e1ba1 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.354510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 341/658] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:58.355073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abe263-5904-494b-ad6c-a030486e1ba1 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.394591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.396860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.466515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abe263-5904-494b-ad6c-a030486e1ba1 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.506548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.507153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.507432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.507708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.508468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.508774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.510206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecd815-7210-4ae8-b3a9-7a71dacbd12a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.513378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 318/659] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:58.514160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.537038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 342/660] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:58.537038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.537038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.537038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.554529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:58.555825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:58.556633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.567637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.629440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da2ab6-8574-4048-9059-b5470d440f57 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.644253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 343/661] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:26:58.645214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.670882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.672068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.672524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.673204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.750752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.752729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.756281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d9ea5-3516-440b-86e9-b2994c2e7cff None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.791569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.792795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.793682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.793827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.794209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.806815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.808387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.811020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:58.825481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:58.826679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:58.827772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:58.827772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:58.827772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:58.833093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 344/662] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:58.833093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.883795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 319/663] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:58.884987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.939996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fda448-57f5-4da1-a481-14becc552936 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.980296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.980924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.981184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.981515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.993143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd96c-dc09-4a0b-9771-3de90af09ee7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.031201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642529f-0198-417b-89b8-0368b260781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.031201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642529f-0198-417b-89b8-0368b260781d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.031201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642529f-0198-417b-89b8-0368b260781d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.031201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642529f-0198-417b-89b8-0368b260781d None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.031593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642529f-0198-417b-89b8-0368b260781d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:59.031593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642529f-0198-417b-89b8-0368b260781d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.031829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642529f-0198-417b-89b8-0368b260781d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.034320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.045035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 320/664] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:59.045907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642529f-0198-417b-89b8-0368b260781d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.062846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.071508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.073097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.155343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642529f-0198-417b-89b8-0368b260781d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.178970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 345/665] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:26:59.178970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.197559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.198559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.198905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.199246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.211843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.213359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:59.228924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:59.230107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:59.230999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:59.230999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:59.230999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:59.230999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 321/666] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:59.231233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.283374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.283577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.291195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d22679-92af-440f-ac45-209d81fcc19b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.328462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.329275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.329647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.330014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.340575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144c4f-1e62-4cc2-9a5b-38deb13c5b98 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.341400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:59.346184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:59.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 346/667] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:59.348948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.384209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.384903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.385182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.385507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.393481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.395580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:59.407386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:59.408455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 322/668] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:59.409351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.464119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb32a1-baaf-437f-8139-bfadcf00f6e2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.501905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.502545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.502827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.503107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.514040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.514664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee7ecb-9123-4f91-a525-4cae2e1d2913 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.515505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.525580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:59.531454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:59.532646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:59.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:59.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:59.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:26:59.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 347/669] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:59.533710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.547541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.560302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.560993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.561339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.561687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.562334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:59.562637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.564061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.581292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 323/670] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:59.581292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.649105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d3e01-f8b6-4fb2-9df4-4c024f133e9c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.686232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.686869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.687212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.687554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.688314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:59.688693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.690130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.690842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bae9f-cd59-40fc-9de2-6202c7f9a63d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.703197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 348/671] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:59.703197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.733662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.735675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.736170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.736503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.737385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:59.737817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.740138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.758285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 324/672] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:59.759251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.772266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.791572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:59.805571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c99d0-5bfc-455c-8ea3-51d337b55580 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.840989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.841684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.842032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.842371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.843080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:59.843451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.844912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.861997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 349/673] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:59.862660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.872507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f75bf7-f6e4-4795-bc80-bbad69da888b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:59.908407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:59.909129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:59.909404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:59.909673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:59.917217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:59.918901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:59.930597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:59.931700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:59.932840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:26:59.936324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 325/674] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:59.937163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:59.975694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e665649-9346-474b-8e9f-73d5ccb91ee2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.007601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.014112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.014741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.015571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.015571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.026210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.030519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.040522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:00.047073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:00.048232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:00.049331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:00.049331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:00.049331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:00.055085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 350/675] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:00.055085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.061728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d600d7e-b8b7-43dc-9332-9435e7c8c21f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.100956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.101721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.102074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.102419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.103141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=505ea6d7aac346e1a604b192a2a0c364) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:00.103446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.104934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.110784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 326/676] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:59 2022] GET /identity/v3/role_assignments?user.id=505ea6d7aac346e1a604b192a2a0c364&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.111613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.185436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb267aff-973a-42a9-b071-0de21b6a08f7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.230528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c138c-5f7b-41f4-ac1c-15e1eeabe334 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.231934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.232609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.232886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.233157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.241361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.242828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:00.256033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:00.257034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:00.257916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 351/677] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:00.259298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.277107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.277840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.277999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.278272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.278970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:00.279707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.280950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3137-8822-4c97-915b-dd824561d708 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.294226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 327/678] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.296271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a37a0-7dbb-42be-96ef-175acead29c9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.296271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a37a0-7dbb-42be-96ef-175acead29c9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.296271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a37a0-7dbb-42be-96ef-175acead29c9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.301680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 328/679] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:00.301680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a37a0-7dbb-42be-96ef-175acead29c9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.311008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.317825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.375487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e23ced-1b11-4515-bb43-9ffad2a54667 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.413822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.414438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.414769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.415048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.415691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:00.415871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a37a0-7dbb-42be-96ef-175acead29c9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.416060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.417645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.431363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 352/680] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.436106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.460472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.461110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.461386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.461659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.462352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:00.462656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.464128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d3063-1c87-486c-88bf-123cb243441d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.476847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 329/681] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.478142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.478423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.478691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.496304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:00.497407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:00.542564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8b3a-f6d9-4a77-bbdd-a2e56f45657c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.544103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.574412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 330/682] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:00.575271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.577465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.588163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.589141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.589587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.590033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.591085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=3671de0f76cc475db615dcc9b351c271) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:00.591598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.593376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.612343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 353/683] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/role_assignments?user.id=3671de0f76cc475db615dcc9b351c271&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.614734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.765287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99368-3614-4148-9bd7-e7ed20f6cefc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.769347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd9561-58da-4462-a69c-d73663caff9f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.812296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.813017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.813291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.813572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.814308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=11df13be0c5543748f98b9ea3d83c3b8) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:00.814669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.817285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.826470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 354/684] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/role_assignments?user.id=11df13be0c5543748f98b9ea3d83c3b8&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.835478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.836540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.837018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.837500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.846551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.848544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:00.861964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:00.862963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:00.864769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:00.872018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 331/685] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:00.873803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.898636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.915874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.983650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:01.007901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ce4dc-e0b3-4db2-b276-080271d6949f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.037705 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899075c-796b-4416-a890-fc816627bbd1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.058818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.059895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.060415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.060887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.061918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:01.062441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:01.064274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:01.080827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.081778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.083253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.083737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.091413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 355/686] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:01.093713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.104641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:01.113420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:01.114490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:01.115851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 332/687] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:01.121170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.240343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:01.265571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9139be-b935-4a95-9756-4104ddbd482c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.294305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f87773-5909-41b8-8f8a-2a41ebae6302 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.330162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.330814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.331060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.331336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.337731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:01.366145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.366772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.367986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.368279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.368965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:01.369280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:01.370682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:01.393151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 333/688] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:01.399970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.508775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:01.512347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:01.546325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b265550-fa6a-46a1-b3c7-3cd4525a8909 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.583830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.584783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.585120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.585398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.597294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:01.599023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:01.616701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:01.617696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:01.619023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:01.619023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:01.619023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:01.619023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 334/689] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:01.624830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.676305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:01.716993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:01.746534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 356/690] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:01.752302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.770260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3224-4fe4-433a-b7ce-ab959ea9a397 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.834654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.835335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.835716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.836132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:01.836888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:01.837288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:01.838756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:01.863633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 335/691] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:01.869365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:01.926910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e49c19-c262-48d0-bbc1-10d2422312bc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:01.981490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:01.982445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:01.983676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:01.984201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.004144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.006826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:02.031621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:02.032676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:02.034175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:02.034175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:02.034175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:02.037735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 357/692] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:02.039329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.039744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.052195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303247c-c974-487e-8653-94d6fda81dd2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.088550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.148696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.149520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.149804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.150123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.153231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222534-3353-4d69-939f-eeefa5d2a313 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.174169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.175631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.188762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.189370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.189650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.189929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.190576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:02.190880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.192327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:02.205172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:02.206222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:02.207898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 336/694] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:27:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:02.211291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 358/694] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:02.212714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.213437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.401656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.405874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63db9cd-df4e-4ba6-b85d-9bb9394810b5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.417159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8726ef-f466-4d9f-a8e7-ddba923799fe None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.443237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.443854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.444177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.444466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.445150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:02.445457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.446862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72210146-6f2c-45fc-a8a7-943b5570657c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.458177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.459106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.459587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.460097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.465013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 359/695] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:02.466354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa801-3e22-4894-8cf9-608696ec7731 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.466637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa801-3e22-4894-8cf9-608696ec7731 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.466909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa801-3e22-4894-8cf9-608696ec7731 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.472346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 360/696] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:02.472346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aa801-3e22-4894-8cf9-608696ec7731 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.473543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.475765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-bfd26542-eb4a-4678-b828-ded15ea32681 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:02.489303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:02.490486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:02.491362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:02.493858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 337/697] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:02.495256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.495742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.496281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.516485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:02.518153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:02.530904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.581094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aa801-3e22-4894-8cf9-608696ec7731 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.608364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 338/698] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:02.609768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.616880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.617493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.617777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.618052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.647754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.690941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.692354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.829038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 361/699] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:27:02.837561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.856818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae27272-feb6-4a3e-852a-e42f615c2396 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:02.933658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:02.934266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.934571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.934844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.936012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:02.936335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:02.937762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:02.951000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 339/700] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:02.956863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:02.958225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:02.972364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.050898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733322b8-fea0-4011-8d0c-6f5e8e848b2d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.099780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.100813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.101367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.101900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.113910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.115781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:03.132890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:03.133868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:03.134751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:03.134751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:03.134751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:03.134751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 362/701] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:03.135572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.137651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45cb5b-c41e-4842-ab92-36ed6bb1b7d0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.178791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.179401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.179683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.179975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.180753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:03.181063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.182488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def71c-d0d3-497e-a84d-7ddd14576d23 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.188676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 340/702] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:02 2022] GET /identity/v3/role_assignments?user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:03.189926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58373f42-6966-431d-b846-44daa4c65123 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.190214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58373f42-6966-431d-b846-44daa4c65123 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.190543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58373f42-6966-431d-b846-44daa4c65123 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.196987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 341/703] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:03.196987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58373f42-6966-431d-b846-44daa4c65123 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.264071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712ef7a-235b-407c-acd4-e58534c53817 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.276667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.294034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.302890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.303668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.304062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.304498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] PATH_INFO: `/v3/services/network` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.305128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.306534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-96eae9bd-79ac-4cf3-a819-95f996336383 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:03.318761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:03.319862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:03.321550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 363/704] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:03.322376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0e56b-bfd1-436a-8b4d-a44bcb43bbfc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.322376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0e56b-bfd1-436a-8b4d-a44bcb43bbfc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.322376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0e56b-bfd1-436a-8b4d-a44bcb43bbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.340672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58373f42-6966-431d-b846-44daa4c65123 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.344427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c0d0e56b-bfd1-436a-8b4d-a44bcb43bbfc None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:03.345644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c0d0e56b-bfd1-436a-8b4d-a44bcb43bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:03.382081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.382710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.383001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.383336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.384444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:03.385417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.387090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89be8a-e53a-4374-96f6-bddf9dcec09a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.404472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 342/705] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:03.405842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfecda1-d781-4847-9034-7fc1af094a0d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.406202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfecda1-d781-4847-9034-7fc1af094a0d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.406857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfecda1-d781-4847-9034-7fc1af094a0d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.407998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 343/706] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:03.408780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfecda1-d781-4847-9034-7fc1af094a0d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.427614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 364/707] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:03.428838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.429132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.429472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.451342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:03.452567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:03.542084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 365/708] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:27:03.543632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.552644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfecda1-d781-4847-9034-7fc1af094a0d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.584662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.626259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.652660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.653284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.653561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.653836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.654554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] RBAC: Adding query filter params (name=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:03.654763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.656150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.661375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 344/709] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:03.666653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.669025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788afe22-9805-4906-80dd-5e6d3aab6995 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.711144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.712296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.712820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.713298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.721191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.722996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:03.735732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:03.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:03.737833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 366/710] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:03.744832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.836053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b0a68-b67e-4fdf-adcc-deaf3f625fbb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.894311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.895483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.895842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.896168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.897235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] RBAC: Adding query filter params (type=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:03.897543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.899318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.912236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 345/711] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:03.912236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.932915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.940948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23cbb3-05fe-4639-88cf-d8a04ecf7525 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.991261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.018864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.019937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.019937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.023087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.043798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2da3d5-8774-4284-85b4-5145f8a5469a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.055462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.056995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.090147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.090687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.090968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.091243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.091972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:04.092297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:04.093900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:04.094813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:04.095704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:04.095704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:04.095704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:04.095882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.095985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 367/712] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:04.095985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.109852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 346/713] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:04.110608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.236024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f70a9-c8f2-48b6-81c7-998eacc59891 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.261043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.305927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.306795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.307508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.308269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.309774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:04.310087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.311471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.327212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9ba50-8db0-4227-82b8-445fbb8a44b4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.331240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 347/714] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:04.331240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.367981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.368877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.369259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.369636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.370333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.371798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:04.381375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:04.382403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:04.383240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:04.387520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 368/715] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 24 16:27:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:04.388493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.404031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.477292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25178c-65ce-400e-ba0f-17bbe01a5701 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.502908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4accfd1-4da4-4774-9e37-e592cd7a47d8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.523222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.524173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.524637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.525655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.526858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=f1ab09f1996244a2b328a887fe91febd) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:04.527402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.530024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.540082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 348/716] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:04 2022] GET /identity/v3/role_assignments?user.id=f1ab09f1996244a2b328a887fe91febd&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:04.540082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.557363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.558257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.559836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.560304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.561160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.564823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.610255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 369/717] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:04 2022] POST /identity/v3/users => generated 254 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:04.619715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.686557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060851d-6b17-41ec-bc38-341e54f4890a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.727034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.727661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.727956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.728243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.728887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:04.729191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.730574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c071f-38d6-4eb5-9cfa-66334e0cb214 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.736143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 349/718] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 16:27:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:04.742087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:04.844181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:04.887360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1070-6d98-4393-a829-b114b23a204e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:04.922701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:04.923373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:04.923655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:04.923996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:04.924644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:04.924950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:04.926464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ff97e-e1dc-47a9-b0b1-4f3dcfbee13f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:04.933512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 370/719] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 16:27:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:05.332654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5be0-1559-4806-8345-0df1e8c1ac1c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.333195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5be0-1559-4806-8345-0df1e8c1ac1c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.333611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5be0-1559-4806-8345-0df1e8c1ac1c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.335323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 350/720] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:05.340868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.341311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.341750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.361022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:05.362435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:05.453153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 371/721] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:05.742085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5be0-1559-4806-8345-0df1e8c1ac1c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:05.875972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5be0-1559-4806-8345-0df1e8c1ac1c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:05.883303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:05.913143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4158ba-b3d7-4f11-a4ea-feda751eb85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:05.913829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4158ba-b3d7-4f11-a4ea-feda751eb85e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.914115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4158ba-b3d7-4f11-a4ea-feda751eb85e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.914391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4158ba-b3d7-4f11-a4ea-feda751eb85e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.925969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4158ba-b3d7-4f11-a4ea-feda751eb85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7d4158ba-b3d7-4f11-a4ea-feda751eb85e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:05.930753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:05.931742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:05.932643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 351/722] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:05.939524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:05.968779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b806-1189-4d73-af26-50ab6ccb618c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.969112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b806-1189-4d73-af26-50ab6ccb618c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.969336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b806-1189-4d73-af26-50ab6ccb618c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.970831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 352/723] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:05.976770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.977374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.977915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.998674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:06.000227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:06.070857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.132059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 353/724] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:06.142720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdc23c-614f-4df9-85d0-3710a4d95a32 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.193622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.194239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.194517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.194792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.195454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:06.195756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.197154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.212421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 372/725] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:06.217853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.349037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.355796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091d589-212e-40f3-88ed-c8eb0689ad5a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.404146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.404789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.405070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.405344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.420302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.421831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:06.439982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:06.441039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:06.442644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:06.442644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:06.442644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:06.445192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 354/726] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:06.457069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.616237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d50bc-6d26-4604-8638-af751cf9a0e3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.644968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.646824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.660672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.661290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.661569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.661841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.663045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:06.663351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.664772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.689945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 373/727] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:06.692941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.793666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6c378-5400-4b78-a327-b29ddfd61c64 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.841255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.841880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.842159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.842435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.843105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.844501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.881929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 355/728] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:06 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:06.882721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632424ca-1bd6-4990-9e0c-3ccdfcdfd5ed None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.936079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.936730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.937011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.937288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.946884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.948396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c5c87732-109a-4bbf-952c-bd2ac7f031d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:06.962172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:06.964332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:06.966503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 374/729] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:06.974897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.001341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.104558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89285e-8969-452a-b072-3927180725c6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.104873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89285e-8969-452a-b072-3927180725c6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.105107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89285e-8969-452a-b072-3927180725c6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.107816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 375/730] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:07.111768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15280-8136-4f03-89be-9728763cdead None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.112089 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15280-8136-4f03-89be-9728763cdead None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.112371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15280-8136-4f03-89be-9728763cdead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.128669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23aaa0-e3d3-48ea-b011-fd9be4515afa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.132614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ce15280-8136-4f03-89be-9728763cdead None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:07.133753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ce15280-8136-4f03-89be-9728763cdead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:07.165504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.166608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.167031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.167885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.169265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:07.169845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.170607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.171484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.195550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 356/731] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:07.200660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.218115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 376/732] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:07 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:07.276265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.315115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18f775-3b37-400d-a5ff-dfbbd43c2306 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.352966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d58aea-73cb-41bc-8815-4972eaaeab6c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.353604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d58aea-73cb-41bc-8815-4972eaaeab6c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.354117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d58aea-73cb-41bc-8815-4972eaaeab6c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.356041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 377/733] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:07.361318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cae43-06b9-4a56-9421-11779c09b246 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.361639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.362075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cae43-06b9-4a56-9421-11779c09b246 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.362267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.362573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.362848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.363756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cae43-06b9-4a56-9421-11779c09b246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.391762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-486cae43-06b9-4a56-9421-11779c09b246 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:07.394043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-486cae43-06b9-4a56-9421-11779c09b246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:07.475708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.478671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.533421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 378/734] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:07 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:07.539969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1278750-88fc-451b-ba9a-ee521881e475 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.539969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1278750-88fc-451b-ba9a-ee521881e475 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.539969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1278750-88fc-451b-ba9a-ee521881e475 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.539969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 379/735] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:07.542825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.543108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.543383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.562257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:07.563430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:07.578862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.626815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 357/736] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:27:07.628192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.648356 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 380/737] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:07 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:27:07.649242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.679204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.753218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e70f3-2c18-4fed-8e21-a5d125cbd287 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.772084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affcbc5-0bb1-42bb-819e-82987319f52f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.829256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.829893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.830169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.830442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.841930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:07.842128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:07.842591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:07.842873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:07.843212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:07.846774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:07.848046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 358/738] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:07.849341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.855879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:07.857442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:07.879641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:07.881028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:07.882183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:07.882183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:07.882183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:07.882183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 381/739] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:07.883306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:07.889520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:07.969860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce5dae-7772-45ed-a8a7-4560de03a051 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:07.980507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.005341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0394c4-23bc-482b-aa51-b8b9b48c9780 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.012142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.012755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.013030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.013299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.013933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.015290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.033459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 359/740] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:07 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:08.034313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.059441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.060422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.060878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.062115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.063368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:08.063862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.066431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.080886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 382/741] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:08.082686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.142292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.163804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9166f-f1b5-4447-9374-5024a3dc0db9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.225153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.226583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.226583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.226583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.227063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:08.227437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.228940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.231767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1e6a3-b832-452d-933b-5fe0f3addd0d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.241573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 360/742] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:08.242405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.282703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.283348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.283628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.283902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.292330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.297317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:08.302489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:08.303439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:08.304329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 383/743] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:08.305025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.386950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2855205-8b1f-4e48-9a33-6a128700c9e0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.421983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d455d-eed8-422d-aced-454aa36c66a8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.424339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.425120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.425489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.426009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.438348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.439971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.448183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:08.461138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:08.462100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:08.463012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:08.463012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:08.463012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:08.466935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 361/744] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:08.467777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.475233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.475808 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.476127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.476404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.483865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.485356 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:08.497925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:08.498853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:08.499760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:08.504661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 384/745] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:08.505376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.568453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.594550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083fe52-8e09-4ffd-98aa-8a6a313f617b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.615335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098997f-c1a5-4706-8c49-b25e8cedd0fa None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.631538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.632184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.632488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.632764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.633438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:08.633753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.635204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.644057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 362/746] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:08.645096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.655853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.656814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.657289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.657766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.658599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:08.659104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.661321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.673886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 385/747] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:08.674974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.716865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.762962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1103df5-266a-4b26-bd5b-6a0af3597023 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.781248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc18f0-ce27-49fc-bed1-ebde8660d251 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.800174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.800370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.802386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.802917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.803449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.804392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:08.804852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.807054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.819537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.820190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.820466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.820853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.826547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 363/748] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:08.827693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.833314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.834179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:08.848842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:08.849801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:08.851389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:08.851389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:08.851389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:08.851389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 386/749] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:08.851389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:08.939594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08709f41-8d20-4b15-b85d-6938722c0ea6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.955509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f3a7-a6c5-4f11-95f9-5634ef561030 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:08.962619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:08.979086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.980169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.980658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.981190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.989329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.991142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:08.993149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:08.993750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.994128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.994317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.995010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:08.995320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:08.999713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.011973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 387/750] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/role_assignments?user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.011973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:09.012872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:09.013808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:09.014785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:09.017408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 364/751] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:09.023411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.048765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.142731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb27c1-be25-4857-a064-3e48fae7e04e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.219576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.221523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.221633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.221907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.223102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:09.223398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.225529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.228474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c5649-3a86-4ae9-82e0-c05494ff3d18 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.239405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 388/752] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.245745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.274514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.275461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.276025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.276508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.277486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:09.278014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.279643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df1429-c000-4583-9748-5d61381d4625 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.313918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 365/753] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.315470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf111f4-b5eb-47d1-815b-f9522ba62888 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.315758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf111f4-b5eb-47d1-815b-f9522ba62888 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.316089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf111f4-b5eb-47d1-815b-f9522ba62888 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.318353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 366/754] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:09.319179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf111f4-b5eb-47d1-815b-f9522ba62888 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.355090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.362919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f9e58-90b4-4d13-95e3-5d61a93d9d51 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.371167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.406274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.407001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.407446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.407791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.416200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.417970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3d8d4e29-7327-4884-bd3a-cb05a7f965b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:09.431317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:09.432546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:09.433714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:09.435088 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 389/755] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:09.436315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.436664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.437001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.454013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf111f4-b5eb-47d1-815b-f9522ba62888 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.456094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:09.457558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:09.501907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.502793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.503252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.503847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.518378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.520926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:09.538511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:09.539494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:09.540407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:09.540407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:09.540407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:09.544021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 367/756] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:09.545055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.551804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 390/757] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:09.551804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.639222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.666743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0fb2c-0830-46cb-86bc-179966425aa3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.713535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.714498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.714993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.715455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.716340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:09.716876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.718480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.719783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.732516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 391/758] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.738760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.739788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4371f12-2de4-454c-baec-c75a381e8a51 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.813394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.813945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.814226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.814508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.815252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:09.815558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.817255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.863971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fe972-0e4e-45c4-bcc0-a62f89c1d59e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.874183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 368/759] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.880885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.903151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.904163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.905794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.906271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.920832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.926669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:09.939195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:09.940131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:09.941007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:09.943754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 392/760] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:09.945335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.960299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.044852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18aec-52c8-4c2b-bc38-21eddb316a0d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.059315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d866-1549-4d93-b74a-b8adca5594bc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.081140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.081667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.082627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.083031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.099307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.100575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.101064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.101633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] PATH_INFO: `/v3/services/network` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.102430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.103025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.105207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:10.116598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:10.117595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:10.118457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:10.121365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 393/761] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:10.122397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.164035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.165496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.198664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.235361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723656a5-46dc-47a9-93b7-89f302d7eedc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.271657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.272320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.272754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 369/762] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:10.273650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.277274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.277631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.278404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:10.278716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.280227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a82b36-f658-4e47-9a14-1414c08989c6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.298528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 394/763] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:10.298528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb931e3-2105-4ad2-8b51-61288bc252a0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.298528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb931e3-2105-4ad2-8b51-61288bc252a0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.298528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb931e3-2105-4ad2-8b51-61288bc252a0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.301782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 395/764] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:10.302635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb931e3-2105-4ad2-8b51-61288bc252a0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.331431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.404414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805f4da-b640-4407-b0bb-9afc917c9fc2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.410349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb931e3-2105-4ad2-8b51-61288bc252a0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.436603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.443483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.444297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.444712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.445123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.445239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.445833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.446113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.446304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] RBAC: Adding query filter params (name=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:10.446409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.446821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.448373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.452939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 370/765] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:10.453821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.461918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.463386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ec1cf75d-3316-4da3-8ad2-98738feb22b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:10.478685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:10.479791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:10.480738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:10.480738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:10.480738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:10.480738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 396/766] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:10.481036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.481324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.481599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.499205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:10.500436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:10.562774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549ab79-3c76-4d81-8b40-bc4afd71c297 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.577127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.578815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 397/767] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:10.579349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.608317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.608979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.609259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.609514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/f1ab09f1996244a2b328a887fe91febd/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.611663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=f1ab09f1996244a2b328a887fe91febd, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.614068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.636034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 371/768] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:10 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/f1ab09f1996244a2b328a887fe91febd/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:10.639036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.765270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50a161-3c75-482c-822b-2d15345e10db None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.786187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.820919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.821979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.821979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.822185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.822835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] RBAC: Adding query filter params (type=network) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:10.823148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.827975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.830528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 398/769] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:10.841315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.854201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69fac1-2ec8-468b-8c50-2301f0a97a67 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:10.901949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:10.902491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:10.903284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:10.903383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:10.904089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:10.904400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:10.905823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:10.919243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 372/770] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:10.920955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:10.956539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:10.981906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fbe0f-735e-496f-b45b-d31f39d07668 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.021748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.022362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.022643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.023291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.023725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] RBAC: Adding query filter params (interface=public, service_id=899d94d2c05a44aabbe38b668a6e2df1, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:11.024109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.025744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.038371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 399/771] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/endpoints?service_id=899d94d2c05a44aabbe38b668a6e2df1&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:11.039312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.061224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb91ae-e399-41f9-a5ee-3dd17202a802 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.105947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.113760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.115859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.116409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.117530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.118649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.121542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:11.136387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:11.137428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:11.138259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:11.141271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 373/772] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 24 16:27:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:11.142233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.162611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d12d3d-ea15-49aa-a439-05e8c6c4d358 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.198657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.199290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.199568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.199841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.206584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.208757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:11.219774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:11.220766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:11.221660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 400/773] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:11.222169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.252824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.260164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b7930-dc45-4d1d-b3e5-565cf398a6f9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.301281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce1b02-6c60-437b-9013-0044ae3cc912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.301921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1b02-6c60-437b-9013-0044ae3cc912 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.302200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1b02-6c60-437b-9013-0044ae3cc912 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.302474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1b02-6c60-437b-9013-0044ae3cc912 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.303090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1b02-6c60-437b-9013-0044ae3cc912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.304566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1b02-6c60-437b-9013-0044ae3cc912 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.316470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 374/774] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:11.317970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6506a-79a1-4893-804f-70f01f996786 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.318257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6506a-79a1-4893-804f-70f01f996786 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.318529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6506a-79a1-4893-804f-70f01f996786 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.323897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 375/775] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:11.324988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6506a-79a1-4893-804f-70f01f996786 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.335271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.342337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8239f9-8f05-476b-8299-430c45c15d9a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.393380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.395165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.395686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.396223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.397135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:11.397737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.399416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc257bb-4d97-4708-bba4-2ff90a1c6996 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.405791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 401/776] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:11.407453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.407993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.408492 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.429908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:11.431473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:11.461928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6506a-79a1-4893-804f-70f01f996786 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.509166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.509781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.510060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.510520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.511097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:11.511361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.513655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.515231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 402/777] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:11.516095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.529008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 376/778] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:11.535618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.571633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.674707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808ca2-13d3-4e9c-8caf-f3612a4e5ad9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.720156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.734645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579dce1b-5005-408b-8623-f7406ed4c6b2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:11.745480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.746089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.746445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.746718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.747496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:11.747808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.749257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.759161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 377/779] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/role_assignments?user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:11.784666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:11.786061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.786341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.786612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.787254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:11.787554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:11.788963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:11.795161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 403/780] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:11.804738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.919860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:11.953837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.978055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c969af-89dc-4750-ab9f-fed544611274 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.018084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.019005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.020344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.020808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.021857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] RBAC: Adding query filter params (interface=public, service_id=6f0596d3933f4a9e9caa99013c6db085, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:12.022485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.024276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.034209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 378/781] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/endpoints?service_id=6f0596d3933f4a9e9caa99013c6db085&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.040919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.085955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37bdd7-56fe-4d55-ad5f-abc12c7c90b6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.125046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.126486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.126952 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.127615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.140090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:12.146522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:12.152490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:12.154505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:12.157749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 404/782] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:12.164669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.187093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.288131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f98f0a-b9bc-4778-9a6b-bfac118bdb15 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.330677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.332394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.333362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.333848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.334726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:12.335259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.337413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.346192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2171a-680c-4a28-a6d2-3ff1b6014f64 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.348550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 405/783] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.355313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.395286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.396052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.396439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.396616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.397223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.398610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.405167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 379/784] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/services => generated 772 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.472295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.508837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be5969-226c-485e-b766-dc06b229b8b4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.565278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.565856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.566166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.567239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.589123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.590591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:12.611594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:12.612646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:12.614064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:12.614064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:12.614064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:12.614064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 406/785] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:12.620653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.639018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.734649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5965bc-b716-4fef-a395-4def06555cf7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.774753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.775702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.776222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.776723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.777597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:12.778105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.779683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb1cf9-bb07-43ad-9e46-2970203ee64d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.793459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 380/786] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.799276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.906933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6715e0-d3f5-4875-a793-b493088cda60 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.938297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.946484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.947450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.948033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.948537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.958436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2225d-cfb0-4d6f-adb1-9fab4d17196c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.958725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2225d-cfb0-4d6f-adb1-9fab4d17196c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.959010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2225d-cfb0-4d6f-adb1-9fab4d17196c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.960664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 381/787] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:12.965333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.966113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.966401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.966676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.967781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:12.985547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:12.986565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:12.987507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:12.990139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 407/788] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:12.996639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.011865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:13.013113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:13.115624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef380b18-a551-4b7f-b274-0ef65af81fe3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.148804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 382/789] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:13.153367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.154114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.154499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.154876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.155722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:13.156229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:13.157890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:13.178929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 408/790] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:13.184700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.187512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:13.318871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c4308-07ef-49eb-b443-a11669054bf6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.358202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.358868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.359112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.359387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.373838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:13.375312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:13.393841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:13.395342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:13.396469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:13.396469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:13.396469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:13.400058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 383/791] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:13.405852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.481489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:13.530032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.558851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57446b0e-dbfe-49d0-b522-040e2bf45365 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.602298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.603166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.603454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.603731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.604401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:13.604703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:13.606532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9405786-4b3f-4837-ab2f-8b042c08867e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:13.625483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 409/792] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:13.625483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb9187-ae5f-4bf4-a6b3-b14f86128487 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.625483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb9187-ae5f-4bf4-a6b3-b14f86128487 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.625483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb9187-ae5f-4bf4-a6b3-b14f86128487 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.625863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:13.625943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 410/793] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:13.626445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb9187-ae5f-4bf4-a6b3-b14f86128487 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.658582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02c8f-67cd-4cc1-82e7-1acc2117073d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.698307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339829c8-d9e9-4391-9026-82ca8d9870cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.699336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339829c8-d9e9-4391-9026-82ca8d9870cf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.699844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339829c8-d9e9-4391-9026-82ca8d9870cf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.700498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339829c8-d9e9-4391-9026-82ca8d9870cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.712527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339829c8-d9e9-4391-9026-82ca8d9870cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-339829c8-d9e9-4391-9026-82ca8d9870cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:13.718318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:13.719274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:13.720186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:13.727637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 384/794] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:13.729390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.729954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.730443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.744267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb9187-ae5f-4bf4-a6b3-b14f86128487 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.750809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:13.752441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:13.753450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:13.780953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.781326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.781658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.781936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.790152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:13.791685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:13.808269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:13.809265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:13.810137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:13.814524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 411/795] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:13.815688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.840853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 385/796] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:13.849283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.923780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:13.924003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08accff-b488-46da-9eee-e09ed6f10f1c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.962733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.963115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.963395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.963669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.964391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:13.964693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:13.966109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:13.989631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 412/797] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.006000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.016071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b25cae-8d62-411c-bc9a-bf98090af3e5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.055060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.056077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.056565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.057085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.057981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.058510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.060280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.069625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.074918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 386/798] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.092668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.153404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77d866-8265-4cf1-84ce-8764e88e0a6c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.188565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.189191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.189466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.189739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.200597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.202035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:14.216898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:14.217867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:14.218783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:14.218783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:14.218783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:14.218783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 413/799] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:14.226161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.296596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4600e-2c0b-404f-b392-f5c9b39a0f6c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.298989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.332923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afca2ae-9b03-4ff5-a272-63378de90e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.333625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca2ae-9b03-4ff5-a272-63378de90e70 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.334022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca2ae-9b03-4ff5-a272-63378de90e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.334309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca2ae-9b03-4ff5-a272-63378de90e70 None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/d1e90301e20d4d4ea7f11c7bef1ef4c8/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.336271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca2ae-9b03-4ff5-a272-63378de90e70 None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=d1e90301e20d4d4ea7f11c7bef1ef4c8, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.337705 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca2ae-9b03-4ff5-a272-63378de90e70 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.342065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.359763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 387/800] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:14 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/d1e90301e20d4d4ea7f11c7bef1ef4c8/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:27:14.359763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db333ab-2435-434e-87f5-fd4aff3849a8 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.359763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db333ab-2435-434e-87f5-fd4aff3849a8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.359763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db333ab-2435-434e-87f5-fd4aff3849a8 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.359763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 388/801] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:14.359763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db333ab-2435-434e-87f5-fd4aff3849a8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.373625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751115a-385c-4c1b-b4c0-24ad8f78f1ff None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.420532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.421166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.421452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.421730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.422389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.422695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.424201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd0597-4073-4cfc-9724-87f9cd6a9e10 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.447847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 414/802] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.449343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.449633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.449915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.469800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:14.471019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:14.510301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db333ab-2435-434e-87f5-fd4aff3849a8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.554643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 415/803] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:14.555535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.558373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.559013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.559293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.559568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] PATH_INFO: `/v3/services/network` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.560190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.561576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:14.574853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:14.575886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:14.576752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 389/804] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:14.584425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.616604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.708436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29bb72-3824-4766-8841-a05bbdb47daa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.760713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.761334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.761612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.761886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.762528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] RBAC: Adding query filter params (name=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.762832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.764304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.774259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 390/805] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:14.776416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.779694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.811609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9bc14-02a5-4eee-b650-1e08bca53488 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.853638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.854293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.854571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.854842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.862626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.864797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-bec1eff9-648c-44fa-9250-3817c245b8f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:14.880263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:14.881253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:14.882169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:14.882706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 416/806] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:14.888119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47dbbd-6689-4dad-a9c4-9d606fc388ba None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.888414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47dbbd-6689-4dad-a9c4-9d606fc388ba None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.888699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47dbbd-6689-4dad-a9c4-9d606fc388ba None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.890185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 417/807] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:14.890741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47dbbd-6689-4dad-a9c4-9d606fc388ba None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:14.907475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:14.921619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fcbe7-588a-4433-b7a7-2d08ac1357df None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:14.961321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:14.961929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:14.962208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:14.962481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:14.963203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] RBAC: Adding query filter params (type=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:14.963414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:14.964809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:14.969633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 391/808] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:14.970401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.007441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47dbbd-6689-4dad-a9c4-9d606fc388ba None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.047022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.053609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f788a-cf0d-4098-92de-f73b1bd98ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.054462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f788a-cf0d-4098-92de-f73b1bd98ecd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.054741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f788a-cf0d-4098-92de-f73b1bd98ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.055009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f788a-cf0d-4098-92de-f73b1bd98ecd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.065532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f788a-cf0d-4098-92de-f73b1bd98ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-dc2f788a-cf0d-4098-92de-f73b1bd98ecd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:15.070862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:15.071786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:15.072948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 418/809] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:15.073642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.073642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.073642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.088733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc910-3305-4973-9656-0e070b789283 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.092864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:15.094055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:15.127334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.127963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.128245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.129248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.130431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:15.130707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.132208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.157289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 392/810] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:27:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.157289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.171886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 419/811] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:15.172817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.182273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.274906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5044c7-448e-4d72-aa0a-6a06da2e6220 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.292962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cddcbe-e3f6-481f-b5c1-8f594666ce52 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.303677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.310088 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.310708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.310993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.311270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.311980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:15.312296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.313667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.328192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 420/812] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.328988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.337960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.339409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.339897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.340858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.342158 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.344219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.373739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 393/813] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:15 2022] POST /identity/v3/endpoints => generated 337 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:15.375403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.474777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64708bc9-08e3-4adf-a861-ffb41c05b0ba None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.521417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.522048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.522329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.522600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.540709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.584609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.610395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.611981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.702536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6be88-8c8a-4efa-be79-57417cc75856 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.727618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 421/814] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:15.728414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.742235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.743149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.743426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.743696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.754897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.756297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:15.771456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:15.772422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:15.778702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:15.778702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:15.778702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:15.778702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 394/815] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:15.778702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.827352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.849570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd258f1-1d61-436a-84d0-cc614c63760d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.879686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7123f-5b08-4375-af69-18109945a0a4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.884805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.885429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.885702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.885970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.886543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.887886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:15.897786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:15.898789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:15.899675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 422/816] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:15.900506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.909457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.923149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.923778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.924086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.924357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.936492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.938089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:15.953887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:15.954865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:15.955752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:15.955752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:15.955752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:15.955752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 395/817] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:15.955752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.015742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cfb4c-e331-48f0-9ab4-d2d2dea81bdc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.055661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.056296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.056573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.056843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.057477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.057773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.059127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.071543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 423/818] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.072378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.077241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.079386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecec0d-5e78-4527-a48e-232efa305162 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.118512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.119129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.119400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.119690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.120426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.120849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.122203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.131720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 396/819] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.131720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.172848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.181557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ee40b-8e1d-46a6-b889-fcd1f9e41709 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.219294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.219916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.220265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.220549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.221191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.221495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.223391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.240021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 424/820] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.240021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.247226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcfe07-d1b8-4884-a672-3946e9f98c7c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.287600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.288634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.288943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.289455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.298972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.301528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.303338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:16.316753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:16.318025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 397/821] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:16.319051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.351838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac2dfd-a346-472e-9828-02373caf3e4d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.385564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.386265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.386612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.386991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.387646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.388042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.389442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.395016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 425/822] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.400310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.429172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.433501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881e94-cd24-406a-b7fd-07f8cea7ae76 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.471101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.472096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.473204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.473530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.481162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.482665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:16.495114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:16.496246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 398/823] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:16.497175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.508670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80232a4-ee0a-42cd-9c6a-df6f93787428 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.534650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.544799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.545410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.545684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.545955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.546681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.546980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.548418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.566773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 426/824] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.567720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.605342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e71afd-8510-4459-b63b-72998855d9b0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.638436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392bd98-d8f2-42ab-903a-aed337ba5bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.638971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392bd98-d8f2-42ab-903a-aed337ba5bee None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.639253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392bd98-d8f2-42ab-903a-aed337ba5bee None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.639529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392bd98-d8f2-42ab-903a-aed337ba5bee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.640215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392bd98-d8f2-42ab-903a-aed337ba5bee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.641574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392bd98-d8f2-42ab-903a-aed337ba5bee None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.661146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 399/825] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:16.662342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f316-677b-4d11-8c9b-b7b940604e9d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.662632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f316-677b-4d11-8c9b-b7b940604e9d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.662907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f316-677b-4d11-8c9b-b7b940604e9d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.664148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 400/826] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:16.664853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51f316-677b-4d11-8c9b-b7b940604e9d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.679161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7050ca-e3be-4632-b8cc-339eeb48759f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.697441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.727412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.728062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.728346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.728622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.729328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.729635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.731090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c4692-a2e9-4595-92b5-22c39a768363 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.747441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 427/827] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:16.749273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.749730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.750113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.774374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:16.776407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:16.799815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51f316-677b-4d11-8c9b-b7b940604e9d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.828131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.850168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.851252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.851540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.851819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.863753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.865286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.867061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 428/828] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:16.867882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:16.888887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:16.889852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:16.890732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:16.890732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:16.890732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:16.893304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 401/829] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:16.900154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:16.978768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:16.996816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506a8af-d550-4933-ae57-ed844c47c510 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.024270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85809853-3cbf-467e-9822-a85d778b7ccb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.043360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.044472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.044902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.045324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.046833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=f1ab09f1996244a2b328a887fe91febd) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.047298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.048951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.058272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 429/830] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/role_assignments?user.id=f1ab09f1996244a2b328a887fe91febd&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.071982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.071982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.071982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.071982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.072469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.072469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.074211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.089967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 402/831] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.095863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.151736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.180300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.236190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35420ca6-59a1-46ab-a617-a5e0ac3dc499 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.281050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.281679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.281953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.282222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.289867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.291353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.302887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae990-2450-4cc2-b969-c6e0e16e7a8e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:17.304934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:17.305899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:17.306788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 430/832] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:17.312906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.337842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.350907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.351636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.352166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.352521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.373409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:17.381246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:17.382193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:17.383055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:17.386635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 403/833] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:17.392472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.436234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.447279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675eaa4-c693-4699-b4e0-b44dd582182a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.496019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.496926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.497615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.498068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.499540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.500134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.502347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.517723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e3bbc-56bc-4e6c-8bcc-7864f90cd130 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.519817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 431/834] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.525067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.557122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.557896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.558334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.558708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.559592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.560033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.561522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.572713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 404/835] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.580643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.641001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.645806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08f4a2-7746-44d9-874a-8a0a91012ded None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.682048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.682990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.683281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.683560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/cd94a4b5dfbf4af6984f8bcf4fb8593b/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.702175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=cd94a4b5dfbf4af6984f8bcf4fb8593b, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.703518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.710859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54221b3-44e3-4c42-81c0-fb44294955db None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.721171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 432/836] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:17 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/cd94a4b5dfbf4af6984f8bcf4fb8593b/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:27:17.765466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.766541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.767126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.767625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.786577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.788514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:17.806929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:17.807933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:17.809243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:17.809243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:17.809243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:17.811251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 405/837] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:17.819036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:17.836366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:17.934104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a3569-466b-460a-b329-591cef9be1fb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:17.971300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:17.971832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.972220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.973488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.974141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:17.974445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:17.975859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:17.992086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 433/838] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.994478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:18.102933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:18.106191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb2fcd-2db5-43a7-ae12-fa7112734614 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:18.144044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:18.145292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.145579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.145856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.154150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:18.155653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:18.168706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:18.169706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:18.170652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 406/839] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:18.176443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:18.251429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:18.294885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f39e-0be5-4bed-aade-3c135d99414a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:18.332900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:18.333641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.333922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.334201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.334928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:18.335234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:18.336671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:18.359918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 434/840] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 16:27:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:18.367037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:18.478196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075269eb-a9b7-4964-b056-cb35e984d067 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:18.514359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77822f4-3d97-4e76-8ce5-51a6c4d6e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:18.514894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77822f4-3d97-4e76-8ce5-51a6c4d6e350 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.515172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77822f4-3d97-4e76-8ce5-51a6c4d6e350 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.515444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77822f4-3d97-4e76-8ce5-51a6c4d6e350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.545675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:18.589176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77822f4-3d97-4e76-8ce5-51a6c4d6e350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:18.590980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77822f4-3d97-4e76-8ce5-51a6c4d6e350 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:18.697001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 407/841] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:18.702473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:18.708113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85136e7-8b15-4dc2-9d76-b7d96098b5c6 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.708707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85136e7-8b15-4dc2-9d76-b7d96098b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.708984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85136e7-8b15-4dc2-9d76-b7d96098b5c6 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.710452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 408/842] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:18.715270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.715556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.715829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.734712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:18.736434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:18.801334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:18.813273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 409/843] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:18.825026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dd1af-ae13-481b-8d58-5ff606eddcc2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:18.850241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:18.861736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:18.862575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.863090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.863637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.874180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:18.875648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:18.892717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:18.893716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:18.895918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:18.895918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:18.895918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:18.895918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 435/844] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:18.900416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.006730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c57cd9-6135-498f-b35e-25c0f09bdd95 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.043059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.043672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.043966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.044253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.044892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:19.045193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.046559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.059345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 410/845] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:19.064663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.099569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.152628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.176783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572940c-4602-4244-8dd8-c3f628d32dcc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.208806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f19022-e9f0-4363-ac25-5a6de1f84e77 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.211632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.212300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.212579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.212855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.219880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.221368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-72416994-0390-45b5-a3b8-c74858c01001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:19.233580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:19.234671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 436/846] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:19.236099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e633-2166-4c5b-9751-0c625095c653 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.236636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e633-2166-4c5b-9751-0c625095c653 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.236636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e633-2166-4c5b-9751-0c625095c653 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.238459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 437/847] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:19.241569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e633-2166-4c5b-9751-0c625095c653 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.255151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.255793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.256545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.256832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.257713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.259355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e2b1f739-c9ae-487a-bcb0-db9f54da4c93 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:19.272077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:19.273064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:19.273885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:19.275752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 411/848] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:19.275752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.276061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.278559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.283562 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.301894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:19.303105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:19.359471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e633-2166-4c5b-9751-0c625095c653 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.371412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 412/849] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:19.372121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.401849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.402802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.403387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.403880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.404822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:19.405363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.407151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.421781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 438/850] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:19.424222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.428584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.485238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860b761-f8ec-4777-a684-44a0c6145b04 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.522688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.523315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.523595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.523866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.524541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:19.524844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.526209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.536859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 413/851] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:19.538150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.561023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6cec-b63a-4468-80da-cb6b9bdd3262 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.564859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.601733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.602413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.602766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.603161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.604105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:19.604725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.606092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.614167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 439/852] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/role_assignments?user.id=d1e90301e20d4d4ea7f11c7bef1ef4c8&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:19.647775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19d687-6316-4670-ab1e-cf24da3ed049 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.659374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.668167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.689128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.690051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.690484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.691654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.692681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:19.693173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.694749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5b34c-111c-4164-bbbc-ced731ba3c9e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.700501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 414/853] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:19.759923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fe8b8-bea3-4ceb-b07a-e2986dc49f89 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.760391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fe8b8-bea3-4ceb-b07a-e2986dc49f89 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.760662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fe8b8-bea3-4ceb-b07a-e2986dc49f89 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.766459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 415/854] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:19.768220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.768561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.768843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.788823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:19.790028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:19.800049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c4953-99db-4cd2-a879-76ca28c11eea None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:19.850861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:19.851505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:19.851787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:19.852746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:19.853433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:19.854850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:19.865524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:19.866490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:19.867395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 440/855] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:19.874882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:19.880556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 416/856] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:19.904736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:19.992573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f9149-da7e-4551-8259-4f13c8ba4c6f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.044739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.045371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.045713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.045992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.046695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:20.047005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.048420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.054528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 441/857] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.061204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.156239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.174955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282e0d-103c-49da-bbeb-7a0218f4bc90 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.177047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.214642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.215252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.215532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.215805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.216476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:20.216776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.218392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.224357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 417/858] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.275356 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.292846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4503bf1-d92d-44ae-9e2d-3ceea9c76fe5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.335272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.335933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.336248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.336528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.349394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:20.354769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:20.355762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:20.356685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 442/859] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:20.364359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.473236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812da8e8-1938-4f5a-bae8-0278947c398b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.509773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.510748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.511232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.511724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.512629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:20.513143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.514718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.528046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 418/860] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.529860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.639825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086be057-2c03-46b4-b6c7-d52e14f67426 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.674740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.675613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.675894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.676201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.687232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.688718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:20.703900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:20.704874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:20.705777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:20.705777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:20.705777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:20.705777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 443/861] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:20.707049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.713409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.748460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.824288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c14652-cc67-4fff-92ff-9ff0046f7e8c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.862377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.863524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.863804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.864189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.864839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:20.865148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.866582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.880537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 419/862] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.886281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.993975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d74cf7-719d-4d10-bb1c-7b5a4b5d24b9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.030428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.030991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.031275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.031542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.040467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.042165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:21.053870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:21.054821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:21.055782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 444/863] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:21.060907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.155177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.168849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d40c-6c7f-4317-8442-cb72911f0752 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.206275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.207054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.207340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.207615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.208790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:21.209101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.210587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.229728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 420/864] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:21.235308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.301060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.347658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb5e8-9788-4569-a2ad-e28ea8e4897c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.385806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb5446-e313-4e7f-900d-329928b5de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.386413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb5446-e313-4e7f-900d-329928b5de73 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.386696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb5446-e313-4e7f-900d-329928b5de73 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.387078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb5446-e313-4e7f-900d-329928b5de73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.460988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb5446-e313-4e7f-900d-329928b5de73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.462438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb5446-e313-4e7f-900d-329928b5de73 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.525533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.571190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 445/865] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:21.576691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.682249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5359532-6028-4019-b3db-df03dfc4d930 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.718034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.718814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.718984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.719260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.729987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.731459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.731664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3cfe5-6a85-4156-a9ff-090b43c968a2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.731894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3cfe5-6a85-4156-a9ff-090b43c968a2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.732217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3cfe5-6a85-4156-a9ff-090b43c968a2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.737547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 446/866] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:21.740235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f9dcd-d562-4f4d-8e4f-291e091a6447 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.740518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f9dcd-d562-4f4d-8e4f-291e091a6447 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.740795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f9dcd-d562-4f4d-8e4f-291e091a6447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:21.748652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:21.749624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:21.750517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:21.750517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:21.750517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:21.755970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:21.755970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 421/867] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:21.760940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.768866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1a7f9dcd-d562-4f4d-8e4f-291e091a6447 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:21.771076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1a7f9dcd-d562-4f4d-8e4f-291e091a6447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:21.857307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 447/868] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:21.867842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447c9f-6f8f-4994-8049-7fe8589cd9f0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.873898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447c9f-6f8f-4994-8049-7fe8589cd9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.873898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85447c9f-6f8f-4994-8049-7fe8589cd9f0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.873898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 448/869] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:21.875378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.875677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.875974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.898874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:21.899380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce67cf0-10be-4a98-b2fd-ca638b19f54d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.900110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:21.945759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.946674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.947135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.947588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.948439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:21.948951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.951047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.972782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 422/870] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:21.973621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.995539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 449/871] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:22.048337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.082990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edd271-29ba-4fa8-9287-87eaca084f97 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.119540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.120188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.120475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.120753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.130160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.131708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.136975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:22.145603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:22.146543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:22.148159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 423/872] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:22.155370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.167446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.257861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56053-4afd-4bae-a254-6a161ace9a4a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.274004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a5c86-bd02-4c22-842a-ae655a1ad4ae None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.290839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.291472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.291748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.292298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.292879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.293198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.294225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d37bd219-63b4-4b67-8b23-2b8e25952393 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:22.303696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:22.305268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:22.306214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:22.309668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 450/873] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:22.310569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.310936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6820c-e117-4f7c-9deb-cc4f507431c9 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.311172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.311262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6820c-e117-4f7c-9deb-cc4f507431c9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.311451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.311557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6820c-e117-4f7c-9deb-cc4f507431c9 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.311725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.312451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:22.312719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 451/874] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:22.312878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.313387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6820c-e117-4f7c-9deb-cc4f507431c9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.314170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.326719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 424/875] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.327526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.400285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.418289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6820c-e117-4f7c-9deb-cc4f507431c9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.425951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381c0cd-0363-42bb-b035-ff15cd2487cf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.451238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.452102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.452383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.452660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.453306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.454660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.459475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.460112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.460392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.460664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.461336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:22.461645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.463122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c29b02-8e2b-4163-8eeb-1d55e38a6ffc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.468870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 425/876] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.470156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1403946-7a29-4517-9af4-11085b735499 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.470445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1403946-7a29-4517-9af4-11085b735499 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.470722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1403946-7a29-4517-9af4-11085b735499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.472728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 452/877] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:22 2022] POST /identity/v3/services => generated 251 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:22.474376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.499575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c1403946-7a29-4517-9af4-11085b735499 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:22.500839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c1403946-7a29-4517-9af4-11085b735499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:22.543463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.587024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 426/878] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:22.593156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20970b0-b9a5-4684-95e8-ca4f104221c9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.596969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1403946-7a29-4517-9af4-11085b735499 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.635791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.637000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.637000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.637201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.638004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:22.638386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.639823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.648408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 453/879] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/role_assignments?user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.724168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1403946-7a29-4517-9af4-11085b735499 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:22.762055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:22.762739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:22.763091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:22.763415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:22.764098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:22.764456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:22.765904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb15f1-2753-4cc8-b5ac-2c5e5c21c8ac None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:22.772056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 427/880] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:22.865894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:22.879402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:22.986870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f4c7e-92c2-48c4-a0f0-fa5f4d7d0bd3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.023044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88c090-24e5-4404-91c2-17318578e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.023675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88c090-24e5-4404-91c2-17318578e876 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.024096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88c090-24e5-4404-91c2-17318578e876 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.024382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88c090-24e5-4404-91c2-17318578e876 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.025043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88c090-24e5-4404-91c2-17318578e876 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.026483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88c090-24e5-4404-91c2-17318578e876 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.047999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 454/881] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:22 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:24.635665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f4c58-8648-4083-bc4d-2ad4f51c46a9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.636264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f4c58-8648-4083-bc4d-2ad4f51c46a9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.636264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f4c58-8648-4083-bc4d-2ad4f51c46a9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.637798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 428/882] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:24.646800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf387318-1b51-4c6b-919e-5bfa46cb192c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.647697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf387318-1b51-4c6b-919e-5bfa46cb192c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.648491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf387318-1b51-4c6b-919e-5bfa46cb192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.670467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bf387318-1b51-4c6b-919e-5bfa46cb192c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:24.672417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bf387318-1b51-4c6b-919e-5bfa46cb192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:24.735473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e48b8-0e5b-44c2-92b0-d3364496351d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.735803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e48b8-0e5b-44c2-92b0-d3364496351d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.736195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e48b8-0e5b-44c2-92b0-d3364496351d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.739300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 429/883] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:24.743705 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.744014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.744292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.762656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:24.763656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:24.765354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:24.775769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 455/884] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:24.777103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae17d-f0e9-48a3-aaf3-8a46f4cd1cb6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.778000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae17d-f0e9-48a3-aaf3-8a46f4cd1cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.786333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae17d-f0e9-48a3-aaf3-8a46f4cd1cb6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.792161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 456/885] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:24.793528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6b95-4593-434f-bec2-01a001d5beb2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:24.793863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6b95-4593-434f-bec2-01a001d5beb2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:24.794237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf6b95-4593-434f-bec2-01a001d5beb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:24.808670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:24.816963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-48bf6b95-4593-434f-bec2-01a001d5beb2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:24.818812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-48bf6b95-4593-434f-bec2-01a001d5beb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:24.868079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 430/886] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:24.919888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 457/887] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:25.120962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.148244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4d8f-cfa3-4496-9266-c263911875cb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.148920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4d8f-cfa3-4496-9266-c263911875cb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.149307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4d8f-cfa3-4496-9266-c263911875cb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.152119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 458/888] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:25.155337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b4d8f-cfa3-4496-9266-c263911875cb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.247763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1edd2-dee7-49bb-8a7f-8156e6bbab16 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.264033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b4d8f-cfa3-4496-9266-c263911875cb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.282739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.283370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.283652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.283926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.284580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.286055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.287207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ea9bb5ce-f5e9-4e7d-862a-124e3e80b88d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:25.295116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:25.296157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:25.297002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 431/889] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:25.297783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.297783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.298056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.299693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.300870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.301142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.301554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.311734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:25.316721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:25.317048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:25.318148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:25.318991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:25.320029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:25.320103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 459/890] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:25.320103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.338948 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.388103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 432/891] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:25.389088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.425691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac7c1-ff8f-4f55-8568-1b6cc00fde2e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.461392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.462010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.462453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.462740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.464219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.465591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.483708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 460/892] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:25 2022] POST /identity/v3/services => generated 241 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:25.484409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.506227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabdaf-644c-4281-8466-4c9b13e0fa98 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.554276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.555011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.555274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.555542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.555995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.556209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:25.556509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.557906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.565253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 433/893] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:25.571910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.592133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.640547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da4cf-843d-4e64-90b4-ee4d27615b58 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.681135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.681763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.681994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.682267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.682923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:25.683216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.684602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.693129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 461/894] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:25.693601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626b5bc-76f0-40ee-8be1-c90b2f5468d5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.694021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.731277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.732263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.732888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.733260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.733901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:25.734204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.735684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.741911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 434/895] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:25.742848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.799262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa2971-a9c0-40d1-8e08-cd48af3f4e24 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.806876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.835739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.837563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.838245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.838569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.838925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.839574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.840982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:25.850379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:25.851553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:25.852714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 462/896] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:25.853690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.866201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c7e8-b74e-4408-81fb-cd6d108ec4e6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:25.905541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:25.906807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.907305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.907670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.919780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:25.921311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:25.937823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:25.938807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:25.939681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:25.939681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:25.939681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:25.944016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 435/897] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:25.944746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:25.969091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd6d5-5102-45bb-bc33-6cda215edb64 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.011433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.012160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.012442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.012875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.013410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] RBAC: Adding query filter params (interface=public, service_id=f2ce0327d27f4f409af9aaf0974e368a, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.013721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.015106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.025361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 463/898] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/endpoints?service_id=f2ce0327d27f4f409af9aaf0974e368a&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.025361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.064386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.065199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c4f8b-84c9-4896-b8df-5ab7a87d7613 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.096650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.103096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.103825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.104206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.104549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.105249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.105623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.107066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.111795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 436/899] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.117710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.135104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efcb16-2225-4f98-9132-2dc53044313e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.172475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.173081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.173470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.173745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.174381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.174680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.176083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.187821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 464/900] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.188803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.231221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58a5fa-2223-48d2-9092-cad0a0ea6b79 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.279025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.279468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.279752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.280056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.280674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.282246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.289326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 437/901] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:27:26.293579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.301639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.301862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d38afa-d87a-43c7-a88d-67f6a3bf4eb6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.344242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.344817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.345099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.345375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.346071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.346333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.348319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.358683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 465/902] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.361485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.392777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.443613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd3ade-5ba0-41b9-84aa-4c05be1d24f3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.481976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.482581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.482867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.483688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.493639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.495753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.504719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5cd3c-f43e-4b8e-ae43-b6dee5bf5966 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:26.508382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:26.509332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:26.510263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 438/903] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:26.515296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.547210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.548108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.548540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.548997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.549925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] RBAC: Adding query filter params (interface=public, service_id=510d746dbf174a26b16de6d933846b10, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.551292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.552960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.558867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 466/904] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/endpoints?service_id=510d746dbf174a26b16de6d933846b10&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:26.567982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.599890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.672525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e87d1-e979-4d9f-8f73-6c40d45e75df None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.694669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b2532-1fa2-4530-9bac-9f140c22ea4c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.696046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.711891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.712284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.712557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.712827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.713558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.713858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.715340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.734126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 439/905] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.735062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.735678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.744003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.744003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.744003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.744003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.744425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.752097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 467/906] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:27:26.877698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474990b-90c1-4fe5-92eb-9443434217d9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.916171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.917037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.917321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.917598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.966803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.991683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.993176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.101153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 440/907] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:27.106529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.196384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.211608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09178ca2-6724-4276-a9e2-66d3bfc2efaf None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.246781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.247413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.247694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.247987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.262062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.264099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:27.280118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:27.281182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:27.282752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:27.282752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:27.282752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:27.282752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 468/908] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:27.287180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.380485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.393140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19154-ef76-458b-9985-05258de8b701 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.431276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.431886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.432189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.432603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.433256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:27.433566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.434939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e4c7-2e49-42a2-9151-758c4e9a10ba None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.447535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 441/909] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:27.453164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.501714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71211d2-2b1f-4aa8-b150-77ebed3f610c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.501935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71211d2-2b1f-4aa8-b150-77ebed3f610c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.502190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71211d2-2b1f-4aa8-b150-77ebed3f610c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.504082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 442/910] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:27.511181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.511710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.512220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.530610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:27.532148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:27.574135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a98d-b8a9-4390-bb05-67041ed18421 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.611973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 443/911] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:27.615082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.615718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.616023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.616311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.623245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.624820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.626474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:27.637047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:27.637968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:27.638942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 469/912] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:27.646478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.728281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.765163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe37c4-1e35-4ea3-a456-ca0fd9674345 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.803823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.804521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.804803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.805081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.805779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:27.806082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.807513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.821631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 444/913] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:27.829039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.904075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.954209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683654ff-fe24-42b1-a6ba-5967e7de004c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.005904 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.006838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.007134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.007408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.008328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] RBAC: Adding query filter params (role.id=d1c9bf9ea3a7404bb72a7b829bf0733e, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:28.008773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.010457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.023979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 470/914] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/role_assignments?user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=d1c9bf9ea3a7404bb72a7b829bf0733e => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.041583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c19e0b-0371-49d4-8c79-6f2d1dccab7d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.075651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.086572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.087274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.087558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.087832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.088466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.089909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:28.101175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:28.102292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:28.103128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 445/915] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:28.108893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.179923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.223501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca03ff7-d0f7-41fc-b4e5-7d5adfd3f77d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.258584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.259231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.259513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.259790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.260455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:28.260757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.262120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfa8a4-1384-40f9-9284-4c00d0c299ba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.267251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 471/916] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.273815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.350378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d92a-6ac6-4992-b478-cb8c7f895f23 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.350668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d92a-6ac6-4992-b478-cb8c7f895f23 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.350951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d92a-6ac6-4992-b478-cb8c7f895f23 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.352854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 472/917] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:28.360551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.360833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.361109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.381276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:28.382646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:28.386457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd72c6-6aa9-4e6d-b9a7-5bfc5d807ca8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.423551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.424256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.424634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.424802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.425942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:28.426356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.427727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.433789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 446/918] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.443725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.473851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 473/919] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:28.532378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.564797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbcd7e-589c-49d1-80fc-e50b55cb42d7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.601831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.602449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.602730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.603008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.603718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] RBAC: Adding query filter params (interface=public, service_id=f66331e018d44448ad9a892acd4dbcaf, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:28.604053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.605504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.611086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 447/920] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/endpoints?service_id=f66331e018d44448ad9a892acd4dbcaf&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:28.617363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.676844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.723270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd5c0-ac3d-4263-8a70-65daf45a9d06 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.759108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa021-b3b8-427b-8607-86e1d816f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.759729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa021-b3b8-427b-8607-86e1d816f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.760038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa021-b3b8-427b-8607-86e1d816f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.760326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa021-b3b8-427b-8607-86e1d816f9a1 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.760496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:28.760953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfa021-b3b8-427b-8607-86e1d816f9a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.762287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfa021-b3b8-427b-8607-86e1d816f9a1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.770424 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 474/921] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:27:28.795847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a82b0e-7924-4b93-9793-7e749f092132 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.796086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a82b0e-7924-4b93-9793-7e749f092132 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.796344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a82b0e-7924-4b93-9793-7e749f092132 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.798168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 475/922] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:28.803976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.804267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.804537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.824282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:28.825522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:28.889794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af4f96-2bf2-4a98-855b-89ceb6bd583c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:28.914390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:28.918563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 476/923] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:28.933369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:28.934415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:28.934929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:28.935204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:28.935850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:28.937244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:28.947766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:28.948850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:28.949707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 448/924] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:28.957101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.012725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.068266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25adf44-c5a4-4653-84c2-e30c61c7ec0c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.103646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.104283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.104560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.104833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.105459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.105762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.107143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.112298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 477/925] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.118617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.207525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.235458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50d655-9f67-4b99-bf54-182136bc28aa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.278056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.278718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.279226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.279511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.280178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.280483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.282328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.290149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 449/926] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.297273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.336619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19930d84-936c-4685-8775-2c79cbb4958f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.350528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.353630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.378838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.379620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.380032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.380405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.381098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.382656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:29.392375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:29.393411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:29.394334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 478/927] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:29.400468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.412452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fade0-9220-47c9-982d-93572de9c345 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.448303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.449275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.449768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.450237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.451109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.452687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.484404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 450/928] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:29.517868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0613-6efc-4abc-933b-dec20668e082 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.567274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.568348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.568840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.569309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.570164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.570664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.572301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.578611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 479/929] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.585701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.704496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa420-dfbd-47a7-b553-e2add168c2e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.708833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.754140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173edad-b787-4843-b430-448889a3d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.755154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173edad-b787-4843-b430-448889a3d468 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.755646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173edad-b787-4843-b430-448889a3d468 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.756158 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173edad-b787-4843-b430-448889a3d468 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.757015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173edad-b787-4843-b430-448889a3d468 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.757560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173edad-b787-4843-b430-448889a3d468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.759133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173edad-b787-4843-b430-448889a3d468 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.765006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 451/930] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.772276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.816520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.881105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8343fe-ece0-42cb-9126-84c3b1fb58ed None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.918114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc164f-9402-4417-92b3-432e741b02d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.919100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc164f-9402-4417-92b3-432e741b02d0 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.920064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc164f-9402-4417-92b3-432e741b02d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.920565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc164f-9402-4417-92b3-432e741b02d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.921449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc164f-9402-4417-92b3-432e741b02d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.923019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc164f-9402-4417-92b3-432e741b02d0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.948564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 480/931] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:30.039375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bfab-a6f5-42f9-a490-fb70cbbd24cd None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.039678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bfab-a6f5-42f9-a490-fb70cbbd24cd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.039936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bfab-a6f5-42f9-a490-fb70cbbd24cd None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.041592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 452/932] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:30.047600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.047926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.048195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.067855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:30.069227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:30.150655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 481/933] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:30.419015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bfab-a6f5-42f9-a490-fb70cbbd24cd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.533653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bfab-a6f5-42f9-a490-fb70cbbd24cd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.570821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2967a-aa63-4354-a571-745155623060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.571880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2967a-aa63-4354-a571-745155623060 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.572425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2967a-aa63-4354-a571-745155623060 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.572905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2967a-aa63-4354-a571-745155623060 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.585586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2967a-aa63-4354-a571-745155623060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-15a2967a-aa63-4354-a571-745155623060 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:30.591036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:30.592011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:30.592875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:30.595499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 453/934] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:30.602237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.625552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.712393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62802807-40bf-4146-b05c-4663b72e3f67 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.750478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.751488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.752109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.752588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.754099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:30.754627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.756235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043963d-aa90-434b-8562-07eb14414200 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.765123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 482/935] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:30.772082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2967a-aa63-4354-a571-745155623060 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.802644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.844660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c058d-b4d3-4508-826d-58c6b6873536 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.844944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c058d-b4d3-4508-826d-58c6b6873536 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.845226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c058d-b4d3-4508-826d-58c6b6873536 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.847357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 483/936] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:30.852428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.852713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.852992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.873229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:30.875022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:30.886897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2967a-aa63-4354-a571-745155623060 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.922142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.922772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.923053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.923339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.934832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.936317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:30.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:30.951477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:30.952414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:30.952414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:30.952414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:30.956407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 454/937] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:30.957202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.965001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 484/938] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:31.054104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.074806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17233f82-936d-4ad8-899d-2dc63973df2c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.111865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.113088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.113426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.113708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.114353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.114655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.116118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.128982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 455/939] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.134496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.150683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.242409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f804d-d255-4ca9-b4a2-dacbec06bf89 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.260614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.280194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.281245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.281718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.282211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.291181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.293269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:31.306683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:31.307727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:31.308710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:31.311507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 485/940] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:31.318605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.380084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f3ed5-3796-4a4d-b31f-62d84ba27fc3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.416216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.424480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.425928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.426468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.427870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.429491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:31.440111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:31.441810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:31.442672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:31.445751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 456/941] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:31.451171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.457491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.461547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded5172-9323-4bf5-ae77-ff78e0317947 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.500529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.501241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.501595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.501944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.503686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.503814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.505436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.530495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 486/942] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.531509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.591170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6689-5e15-4a0f-9fcb-c36cf71060d9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.643048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.643818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.644130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.644886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.645975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.646424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.647933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f80d0-8351-48b3-bb0d-6a649208ae67 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.648426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c60c3-82d7-47e2-b64d-b40ba0a630c5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.654219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 457/943] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.657449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb040b-6efe-43e4-b961-c6fa7f1348ef None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.657806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb040b-6efe-43e4-b961-c6fa7f1348ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.658160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb040b-6efe-43e4-b961-c6fa7f1348ef None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.660081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 458/944] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:31.660856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb040b-6efe-43e4-b961-c6fa7f1348ef None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.685455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.693095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.693728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.694011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.694290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.704858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.706361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-dbbb9dd4-df75-4a8b-aac9-aa8af6e0d549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:31.721514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:31.723042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:31.723978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:31.723978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:31.723978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:31.723978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 487/945] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:31.724836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.725118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.725395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.742094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.744095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:31.745529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:31.778597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb040b-6efe-43e4-b961-c6fa7f1348ef None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.814085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.814731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.815013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.815291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.816610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.816916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.818297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.821707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 488/946] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:31.822720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.823993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 459/947] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:31.829000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:31.932661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d3534-77f2-4ad3-a191-d3369ede667f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.943534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0405176-d867-4fe1-a06c-83b219e28347 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:31.947108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.973691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.974346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.974707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.975055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.975769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:31.976175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.977618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.980340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:31.986476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0046d3-19ea-49d8-a012-b2f8ebd240f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:31.987220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0046d3-19ea-49d8-a012-b2f8ebd240f2 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:31.987983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0046d3-19ea-49d8-a012-b2f8ebd240f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:31.988851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0046d3-19ea-49d8-a012-b2f8ebd240f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:31.989993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0046d3-19ea-49d8-a012-b2f8ebd240f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:31.991460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0046d3-19ea-49d8-a012-b2f8ebd240f2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:31.991747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 489/948] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.002848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.021492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 460/949] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:32.041475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbacfd-fe97-4285-8ea9-9b50211087b5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.042022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbacfd-fe97-4285-8ea9-9b50211087b5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.042300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbacfd-fe97-4285-8ea9-9b50211087b5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.043925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 461/950] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:32.049938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.050318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.050600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.070780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:32.072216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:32.152776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6056ae0-d02b-46d5-92e2-b72f9df6adc2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.170068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 462/951] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:32.180230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.193852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.194503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.194785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.195059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.203386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.205615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:32.220430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:32.221332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:32.222367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:32.223558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 490/952] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:32.229330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.267641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.318214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b45295-8eea-45d0-876b-9f2e3381fdc2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.344160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.344835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4594f-7557-4b16-9c89-005155b4a825 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.364725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.365366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.365645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.365917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.366554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.368032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.370694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING keystone.server.flask.common [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 16:27:32.384496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.385639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.386146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.386640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.387671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:32.388208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.390404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.393107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.sql.core [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:27:32.393107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 16:27:32.393107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [parameters: {'id': '0a35208aaeb84cadbfbbedc3316afad6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 16:27:32.393107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81688) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:27:32.411514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 491/953] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:32.412564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a35208aaeb84cadbfbbedc3316afad6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a35208aaeb84cadbfbbedc3316afad6. Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:27:32.413879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 16:27:32.414695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application yield resource Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise self.value Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application session.commit() Aug 24 16:27:32.415653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:27:32.416593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:27:32.418199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:27:32.419713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [parameters: {'id': '0a35208aaeb84cadbfbbedc3316afad6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.421617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:32.423291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a35208aaeb84cadbfbbedc3316afad6. Aug 24 16:27:32.426963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.429155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 463/954] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Aug 24 16:27:32 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:32.435203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.527789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.538115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184fb0b-8dd3-4376-be72-38e0ee472b66 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.555684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531caa-ec7f-42a7-a05b-46f61f71359c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.574848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.575442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.575725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.576027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/cd94a4b5dfbf4af6984f8bcf4fb8593b/roles/d1c9bf9ea3a7404bb72a7b829bf0733e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.578211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=cd94a4b5dfbf4af6984f8bcf4fb8593b, role_id=d1c9bf9ea3a7404bb72a7b829bf0733e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.579709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.594100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.594735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.595019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.595282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.604316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 492/955] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:32 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/cd94a4b5dfbf4af6984f8bcf4fb8593b/roles/d1c9bf9ea3a7404bb72a7b829bf0733e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:27:32.604316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.612491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.614846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:32.629358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:32.630576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:32.631749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:32.637282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 464/956] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:32.646205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.684836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.753005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db458b-52f5-46a6-bcf7-18a3de21ed5a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.774573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de00c-25e3-4dfe-b9b5-8f55799c7f59 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.803866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.804569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.804834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.805105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.805689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.807043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.815085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.815807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.816185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.816522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.817264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:32.817637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:32.819025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:32.820321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:32.821505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:32.822404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.822488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 493/957] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:32.828938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.842538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 465/958] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.852616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.950269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a27ac-dbf9-4838-ace7-903eeafb3089 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.992163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.993461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.993913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.994345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.995221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:32.995671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.997287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1ec86-7424-477d-8260-e17e3dc5cb2e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.002920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 494/959] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.009572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.127095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.138996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b65ae-be75-4937-b314-1b938a9c0887 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.177209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.177993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.178382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.178777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.179525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:33.179935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.181435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58d86e-e7d4-425f-9c08-07d301cfcee4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.186972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 466/960] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:34.103177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec193-c2d3-4cde-90f3-f56034eab735 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.103736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec193-c2d3-4cde-90f3-f56034eab735 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.103736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec193-c2d3-4cde-90f3-f56034eab735 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.105567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 495/961] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:34.111556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4234-5c48-4d7f-a8c5-94115aaf49c8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.112112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4234-5c48-4d7f-a8c5-94115aaf49c8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.112112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4234-5c48-4d7f-a8c5-94115aaf49c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.131684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c50a4234-5c48-4d7f-a8c5-94115aaf49c8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:34.133320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c50a4234-5c48-4d7f-a8c5-94115aaf49c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:34.212070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 467/962] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:34.480087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ec193-c2d3-4cde-90f3-f56034eab735 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.569605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:34.591182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ec193-c2d3-4cde-90f3-f56034eab735 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:34.605914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fbef2-9ac0-4abc-a8a7-09312e6ada20 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.606230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fbef2-9ac0-4abc-a8a7-09312e6ada20 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.606462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fbef2-9ac0-4abc-a8a7-09312e6ada20 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.608136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 468/963] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:34.613891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.614171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.614439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.627604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:34.628932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.629214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.629491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.630084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:34.631453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:34.634215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:34.635623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:34.640617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:34.641614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:34.642437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:34.646786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 496/964] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:34.648246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.719980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 469/965] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:34.754034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26cf0e-2207-45f4-9a80-40fd25f7c723 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:34.790129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:34.790735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.791016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.791291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.792202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:34.792506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:34.793864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:34.803553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 497/966] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:34.805039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.828325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:34.912456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c6bd-f2e7-468e-92fd-9b4b62fb3882 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:34.947706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:34.948346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:34.948622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:34.948894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:34.949587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:34.949896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:34.951256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:34.961368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 470/967] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:34.962721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:34.980316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.021701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.083770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2606c-9320-4b11-b7ae-6ff7b635303b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.114420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.121052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.121669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.121948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.122226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.122934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] RBAC: Adding query filter params (interface=admin, service_id=f66331e018d44448ad9a892acd4dbcaf, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:35.123243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.124636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a0e07-8f7f-48b5-925f-ef8daa3c5cd1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.130840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 498/968] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Aug 24 16:27:34 2022] GET /identity/v3/endpoints?service_id=f66331e018d44448ad9a892acd4dbcaf&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:35.132033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dbb56-fb21-41fb-81ad-855b80ff418f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.132317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dbb56-fb21-41fb-81ad-855b80ff418f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.132594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dbb56-fb21-41fb-81ad-855b80ff418f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.133943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 499/969] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:35.137477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dbb56-fb21-41fb-81ad-855b80ff418f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.146751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5349746-e945-4ec9-8a04-d57998ad194f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.181972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cf2a1-3bbe-4d78-9268-396cce7a40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.182604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cf2a1-3bbe-4d78-9268-396cce7a40d9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.182882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cf2a1-3bbe-4d78-9268-396cce7a40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.183152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cf2a1-3bbe-4d78-9268-396cce7a40d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.193277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cf2a1-3bbe-4d78-9268-396cce7a40d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f83cf2a1-3bbe-4d78-9268-396cce7a40d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:35.197992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:35.198953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:35.199801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 471/970] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:35.200604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c185a-ff45-40c4-aad7-894d9d9f9b3d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.200604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c185a-ff45-40c4-aad7-894d9d9f9b3d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.200928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c185a-ff45-40c4-aad7-894d9d9f9b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.218135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.220626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bb6c185a-ff45-40c4-aad7-894d9d9f9b3d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:35.222449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bb6c185a-ff45-40c4-aad7-894d9d9f9b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:35.250762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dbb56-fb21-41fb-81ad-855b80ff418f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.283538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.284783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.285068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.285342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.285940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.287299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.288877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 472/971] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:35.290116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574e213-6c1e-4c07-bea5-ff4f97b3cc84 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.290401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574e213-6c1e-4c07-bea5-ff4f97b3cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.290674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574e213-6c1e-4c07-bea5-ff4f97b3cc84 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.304080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 500/972] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:27:35.304080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.304348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 473/973] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:35.304348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.304348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.304348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.321198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:35.322439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:35.405738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 474/974] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:35.406301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.436739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715bd4b-e9bb-456d-b2f5-f1aaed67c310 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.475689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.476352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.476637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.476918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.477598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:35.477906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.479394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.492287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.493668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 501/975] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:35.495200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.563279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.599445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab895559-25de-4d7d-aee9-a5d98c471383 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.634869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.635742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.635856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.636128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.647211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.648742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:35.663538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:35.664580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:35.665472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:35.665472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:35.665472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:35.669845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 475/976] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:35.671001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.675906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdfadb-f8c6-4251-9011-5045c1e9129d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.712334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.712942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.713225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.713504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.714152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.715565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.718202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING keystone.server.flask.common [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 16:27:35.726896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.732125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.sql.core [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:27:35.732125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 16:27:35.732125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [parameters: {'user_id': '4a21f4225590451cb8b1354e5fe8486f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 16:27:35.732125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81687) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result.read() Aug 24 16:27:35.739737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:27:35.740664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application yield resource Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise self.value Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application session.commit() Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:27:35.742227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:27:35.743173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:27:35.745270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result.read() Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:27:35.748348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [parameters: {'user_id': '4a21f4225590451cb8b1354e5fe8486f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:35.750359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 16:27:35.752418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 502/977] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 16:27:35 2022] POST /identity/v3/users => generated 160 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:35.756174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.793029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c6e15-4f03-4e2f-9adb-d4602627b870 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.807700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:35.836535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.837066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.837367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.837725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.838489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:35.838794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.840179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.858332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 476/978] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:35.858332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.858594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5a3f8-5c1f-4941-9748-7072ec59b330 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:35.892938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:35.894090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:35.894499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:35.894994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:35.895407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:35.897016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:35.915805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 503/979] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:35 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:35.917284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:35.994380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53888ca-2170-49d9-a126-3c7f16547bc6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.015650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.069362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.070001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.070281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.070557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.073130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319182e6-9a20-4e16-98f7-7f92d508d7f4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.079346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.080917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:36.094917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:36.096000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:36.097011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 477/980] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:36.098397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.101924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.123509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.124230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.124583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.125028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.137087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.138979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:36.152036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:36.153014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:27:36.153930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:36.155918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 504/981] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:36.163319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.241396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b3602-3946-4817-a443-e784eeb8e56e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.267775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadc412-2013-49e9-825b-d32c4281464a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.277856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.278819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.279306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.279800 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.280746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:36.281274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.282901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.306432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.307078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.307358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.307634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.308405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:36.308716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.310138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.329229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 478/982] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:36.338450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 505/983] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:36.339998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.380391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.466791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df33bb-9cb0-406b-9741-29cfdb6e001d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.505819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.506435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.506716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.506992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.580401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.582492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.608143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.688628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 479/984] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:36.694646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.737707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:36.800793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f339f-f41d-4b7e-8a58-eb865b6e78ce None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:36.835000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:36.835634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:36.835914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:36.836227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:36.846577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:36.848736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:36.864762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:36.865741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:36.866671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:36.866671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:36.866671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:36.870524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 506/985] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:36.871951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:36.976323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345e698-4c4f-48f3-89b4-e208e0faa8c2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.005437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.017671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.018370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.019871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.020217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.020890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:37.021202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.022611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.042213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 480/986] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:37.043566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.140400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.151685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc755b7-2307-4411-aae0-d8370b1df204 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.186975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.187578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.187912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.188213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.195235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.196696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e536ef37-007c-409a-b0d6-946a2cb24596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:37.208331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:37.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:37.215294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 507/987] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:37.215813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.302719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21445c14-0872-406a-aaa3-9c23f5837150 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.302912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21445c14-0872-406a-aaa3-9c23f5837150 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.303191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21445c14-0872-406a-aaa3-9c23f5837150 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.304696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 508/988] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:37.309463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.309749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.310027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.322254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baffb4-02aa-44b9-aa95-a746f4c4dd4a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.328993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:37.330505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:37.361663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.362547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.363144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.363423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.364192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:37.364546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.366642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.381633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 481/989] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:37.387571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.401362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 509/990] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:37.439208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.497517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c893915-c20a-41a8-9a85-3e999031c360 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.532795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.533399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.533678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.534108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.535342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] RBAC: Adding query filter params (role.id=d1c9bf9ea3a7404bb72a7b829bf0733e, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:37.535651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.537114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.543569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 482/991] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/role_assignments?user.id=cd94a4b5dfbf4af6984f8bcf4fb8593b&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=d1c9bf9ea3a7404bb72a7b829bf0733e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:37.671519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.759124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:37.781661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34722077-3e95-4cde-a816-ac60571680b2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.830703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:37.832225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.832639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.833046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.834000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:37.835750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-273a85ae-f900-4bec-9cb1-2ec12a063cf7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:37.848419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:37.849432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:37.850279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 510/992] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:37.857070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:37.910824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f9e5f-cace-4418-b023-9d6b54c24c31 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.911110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f9e5f-cace-4418-b023-9d6b54c24c31 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.911383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f9e5f-cace-4418-b023-9d6b54c24c31 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.914023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 511/993] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:37.919355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:37.919635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:37.919912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:37.940175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:37.941500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:37.966234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79660850-071f-46c1-a5c6-32677388d2a2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:37.977234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.010906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.011525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.011840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.012155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.012796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:38.013100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.014470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.016147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 512/994] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:38.024154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 483/995] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:38.026081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.126513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c7cbe-c876-4e48-9692-82f3ac0ce807 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.161506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.162528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.163006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.163473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.164632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:38.165148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.167843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.179164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 484/996] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:38.180852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.260401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.287985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f392f-6a4d-46ed-bd2d-cbd747384bde None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.288610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.324028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.324658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.324841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.325125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.325407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.326117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.327748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.355977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 513/997] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:38.410556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba81d3-f05c-4f72-86a1-0c695c397778 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.455143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.455806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.456367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.456954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.457181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.458579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:38.468163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:38.469238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:38.470082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:38.472102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 485/998] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:38.475937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.599579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdd1d3-557f-4cc1-bd0e-b2125fc69668 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.640049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.640660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.640939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.641212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.641849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:38.642151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.644270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.645997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.650589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 514/999] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:38.656080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.759269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:38.766022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f871c14-105c-46cf-93d5-d0f8825f8973 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.801365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.801981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.802260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.802531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.803159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:38.803461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.804842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.809972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 486/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:38.815719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:38.919088 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160d86d-13f3-4b60-80a3-c4736ca82864 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:38.954644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:38.955255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:38.955537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:38.956347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:38.957137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] RBAC: Adding query filter params (interface=public, service_id=86b93b70245b4be98326b13d9f61a5c8, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:38.957446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:38.958871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677b321-d8eb-47a3-9808-d6f704aa8cc8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:38.968508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 515/1001] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/endpoints?service_id=86b93b70245b4be98326b13d9f61a5c8&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:38.970096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:39.099272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e6375-01cd-490f-8c52-1e651e33611f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:39.134537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cc601-e5ee-468f-b3d9-81ff1df5c0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:39.135138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cc601-e5ee-468f-b3d9-81ff1df5c0dc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.135415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cc601-e5ee-468f-b3d9-81ff1df5c0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.135685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cc601-e5ee-468f-b3d9-81ff1df5c0dc None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.136317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cc601-e5ee-468f-b3d9-81ff1df5c0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:39.137682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cc601-e5ee-468f-b3d9-81ff1df5c0dc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:39.145155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 487/1002] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:38 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:27:39.553657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723e3cc-3932-4452-9a03-482d65b869b0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.554188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723e3cc-3932-4452-9a03-482d65b869b0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.554188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723e3cc-3932-4452-9a03-482d65b869b0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.555760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 516/1003] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:39.560770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:39.561171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:39.561365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:39.579571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:39.581298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:39.600231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:39.651980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 488/1004] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:39.928532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723e3cc-3932-4452-9a03-482d65b869b0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:39.992987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.039648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723e3cc-3932-4452-9a03-482d65b869b0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.075539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.076211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.076493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.076769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] PATH_INFO: `/v3/services/image` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.077351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.078706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:40.088357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:40.089366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:40.090204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 517/1005] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 24 16:27:39 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:40.097477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.200267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae55d1-a960-4101-b3e3-53744f6d209e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.235064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.235673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.235978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.236259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.236893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] RBAC: Adding query filter params (name=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:40.237724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.239181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cc5e1-66ac-4bbf-9fbd-8b64b747a748 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.248650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 489/1006] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:40.250237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.317003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac186de-f8fc-4c83-a31f-0b8aa06b3880 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.317322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac186de-f8fc-4c83-a31f-0b8aa06b3880 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.317560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac186de-f8fc-4c83-a31f-0b8aa06b3880 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.322713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 490/1007] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:40.323904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.324216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.324492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.342352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:40.343538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:40.358026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.364486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33142-4933-447b-af38-62c40182a9ee None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.392297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:40.401617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.402225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.402509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.402784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.403419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:40.403729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.405136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.410042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 518/1008] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:40.429916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 491/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:40.697732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.800537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1807-81ff-435d-98ac-3c85816457e3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.835850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc90a95-aad9-4492-bbe0-5716baee8f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.836513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc90a95-aad9-4492-bbe0-5716baee8f72 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.836797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc90a95-aad9-4492-bbe0-5716baee8f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.837071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc90a95-aad9-4492-bbe0-5716baee8f72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.838171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc90a95-aad9-4492-bbe0-5716baee8f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.839586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc90a95-aad9-4492-bbe0-5716baee8f72 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:40.841351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 519/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:40.846914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:40.950881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4383fe-a75d-40b8-9dec-d21c7b867318 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:40.985019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe5e41-e12d-4aac-ac76-75920ce12bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:40.985628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe5e41-e12d-4aac-ac76-75920ce12bdc None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:40.985906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe5e41-e12d-4aac-ac76-75920ce12bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:40.986179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe5e41-e12d-4aac-ac76-75920ce12bdc None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:40.986834 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe5e41-e12d-4aac-ac76-75920ce12bdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:40.988807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe5e41-e12d-4aac-ac76-75920ce12bdc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.013280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.026967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 492/1011] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:40 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:41.130617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93754b72-5a31-4275-b73c-f90434dd2895 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.130986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93754b72-5a31-4275-b73c-f90434dd2895 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.131178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93754b72-5a31-4275-b73c-f90434dd2895 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.132775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 520/1012] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:41.138287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.138569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.138839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.160203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:41.161896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:41.236302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 493/1013] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:41.514353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93754b72-5a31-4275-b73c-f90434dd2895 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.586544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.631012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93754b72-5a31-4275-b73c-f90434dd2895 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.665930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.666827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.667107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.667378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.667978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.669344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:41.678664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:41.679649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:27:41.681367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:41.684813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 521/1014] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:41.685869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.789563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9da69-0c7e-4469-94dc-1c4872d7cca5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.818636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.825397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.826010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.826287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.826558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.827186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] RBAC: Adding query filter params (name=compute) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:41.827488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.828875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:41.834535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 494/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:41.840137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:41.947352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc36d-b8b4-4f02-9286-9156e4773b39 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:41.971286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:41.986782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:41.987399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:41.987683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:41.987975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:41.988961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:41.989483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:41.990859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead7f24-ba3f-4817-a1fc-20fad19d7b86 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.003915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 522/1016] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 16:27:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:42.004127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.109975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a842ce-b42d-4636-8bd8-64cda4969fec None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.149819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eaa3d9-c619-47d9-95f6-a5b09d90969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.150432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa3d9-c619-47d9-95f6-a5b09d90969c None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.150709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa3d9-c619-47d9-95f6-a5b09d90969c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.150979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa3d9-c619-47d9-95f6-a5b09d90969c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.151702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eaa3d9-c619-47d9-95f6-a5b09d90969c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:42.153150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eaa3d9-c619-47d9-95f6-a5b09d90969c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.183978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 495/1017] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:42.318060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4792de5-da7f-45d1-8a16-656489d6e21e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.319107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4792de5-da7f-45d1-8a16-656489d6e21e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.319107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4792de5-da7f-45d1-8a16-656489d6e21e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.319990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 523/1018] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:42.325485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3498f5-7776-498a-b017-07a4180062da None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.325772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3498f5-7776-498a-b017-07a4180062da None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.326051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3498f5-7776-498a-b017-07a4180062da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.345351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-fd3498f5-7776-498a-b017-07a4180062da None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:42.346582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fd3498f5-7776-498a-b017-07a4180062da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:42.400756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:42.430994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 496/1019] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:42 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:42.701466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4792de5-da7f-45d1-8a16-656489d6e21e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:42.809081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4792de5-da7f-45d1-8a16-656489d6e21e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:42.843992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6372fc-050c-49ff-ba0a-eb13d0188642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:42.844622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6372fc-050c-49ff-ba0a-eb13d0188642 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:42.844870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6372fc-050c-49ff-ba0a-eb13d0188642 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:42.845139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6372fc-050c-49ff-ba0a-eb13d0188642 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:42.845795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6372fc-050c-49ff-ba0a-eb13d0188642 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:42.847220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6372fc-050c-49ff-ba0a-eb13d0188642 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:42.866568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 524/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:42 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:43.113244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044b50e-d216-4ab6-b5de-f13204c2357d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.113774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044b50e-d216-4ab6-b5de-f13204c2357d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.113774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044b50e-d216-4ab6-b5de-f13204c2357d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.115316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 497/1021] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:43.120400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.120693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.120974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.138805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:43.140041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:43.184032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:43.215991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 525/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:43.488032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044b50e-d216-4ab6-b5de-f13204c2357d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:43.591102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044b50e-d216-4ab6-b5de-f13204c2357d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:43.624612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacd35e-1a48-4da5-bef1-4aa9f74227b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:43.625241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd35e-1a48-4da5-bef1-4aa9f74227b9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.625520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd35e-1a48-4da5-bef1-4aa9f74227b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.625795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd35e-1a48-4da5-bef1-4aa9f74227b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.626907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacd35e-1a48-4da5-bef1-4aa9f74227b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:43.628888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacd35e-1a48-4da5-bef1-4aa9f74227b9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:43.630668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 498/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:43.636127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:43.735543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5904c-458b-478d-8d0a-d4dd1b62b70b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:43.770999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73b6b-59c8-4d82-bcf5-7accbe136eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:43.771525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73b6b-59c8-4d82-bcf5-7accbe136eba None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:43.771807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73b6b-59c8-4d82-bcf5-7accbe136eba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:43.772114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73b6b-59c8-4d82-bcf5-7accbe136eba None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:43.772776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73b6b-59c8-4d82-bcf5-7accbe136eba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:43.774192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73b6b-59c8-4d82-bcf5-7accbe136eba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:43.803976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 526/1024] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:43 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:44.245969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12616c-e991-48b4-8dba-937bb4eb3f17 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.246508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12616c-e991-48b4-8dba-937bb4eb3f17 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.246508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12616c-e991-48b4-8dba-937bb4eb3f17 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.247905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 499/1025] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:44.253310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.253712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.253926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.272681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:44.273902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:44.297428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:44.345163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 527/1026] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:44.616970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12616c-e991-48b4-8dba-937bb4eb3f17 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:44.665925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:44.726619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12616c-e991-48b4-8dba-937bb4eb3f17 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:44.762206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97478da9-732c-46a4-bbcb-b789e1c0b5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:44.762840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97478da9-732c-46a4-bbcb-b789e1c0b5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.763120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97478da9-732c-46a4-bbcb-b789e1c0b5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.763393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97478da9-732c-46a4-bbcb-b789e1c0b5e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.774362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97478da9-732c-46a4-bbcb-b789e1c0b5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-97478da9-732c-46a4-bbcb-b789e1c0b5e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:44.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:44.780223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:44.781079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:44.782333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 500/1027] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:44.786698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:44.800976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62e0dc-47cf-4532-b496-9720538673e5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.801262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62e0dc-47cf-4532-b496-9720538673e5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.801540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62e0dc-47cf-4532-b496-9720538673e5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.802882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 501/1028] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:44.807776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.808091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.808368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.828955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:44.830148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:44.894259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e874fe-64ca-4f28-b99d-faf5d0eac388 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:44.903361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 502/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:44.930960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:44.931596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:44.931880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:44.932190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:44.932882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:44.933190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:44.934602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:44.944292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 528/1030] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:44.949760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:44.987367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.067633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28df92-27e6-46ad-8a29-f0f56318464f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.102740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.103364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.103637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.103904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.114420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.115849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:45.130855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:45.131819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:45.137094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:45.137094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:45.137094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:45.137094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 503/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:45.139396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.163132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.172334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.248188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b130b-485f-41bc-8c28-c47de47aa6ba None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.265288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.282778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20d06-2574-43b3-b6b9-3f216c707bf1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.284292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.285370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.285370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.285661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.286384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:45.286686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.288066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.301313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 529/1032] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:45.306858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.320863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.321813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.322094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.322369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.323238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.324655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:45.336190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:45.337227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:45.338162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 504/1033] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:45.345425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.406300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.429419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dee02-cba3-4439-9581-0a9deddfb079 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.469211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.469848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.470130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.470412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.473572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6bcf2-001c-4065-981e-ea41433b6cd5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.478579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.480058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:45.492890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:45.493842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:45.494780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 530/1034] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:45.499641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.517245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.518376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.519052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.519658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.520787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] RBAC: Adding query filter params (name=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:45.521242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.522833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.534267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 505/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:45.535341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.553782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.607639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f77c1-4a4c-44e6-8574-7fc2009e0d7c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.642442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.643804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.644122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.644407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.645310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:45.645625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.647372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.648656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb1cea-dcfa-40d1-8ab0-53c2c1e81336 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.657832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.671150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 531/1036] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:45.672341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.687040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.687544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.687826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.688131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.688774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] RBAC: Adding query filter params (type=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:45.689192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.691030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.703394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 506/1037] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:45.704971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.785743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3980-f882-4e0a-9743-927992f21d34 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.790477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.813985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e0032-e991-43ec-bf3a-1c18e9702e75 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.820855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779e1bf-e953-4217-8f47-2587e39f8198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.821461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e1bf-e953-4217-8f47-2587e39f8198 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.821745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e1bf-e953-4217-8f47-2587e39f8198 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.822025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e1bf-e953-4217-8f47-2587e39f8198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.848648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:45.849253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.849536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.849812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.850511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] RBAC: Adding query filter params (interface=public, service_id=fb11d9adb8e64054b4b1497ac16b4778, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:45.850820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.852211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d204a-bf0a-4acb-9cff-337b209481b1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.857116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 507/1038] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/endpoints?service_id=fb11d9adb8e64054b4b1497ac16b4778&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:45.858328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff612-9a57-4504-9018-8742bdaca871 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.858615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff612-9a57-4504-9018-8742bdaca871 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.858891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff612-9a57-4504-9018-8742bdaca871 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:45.863981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 508/1039] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:45.863981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff612-9a57-4504-9018-8742bdaca871 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:45.891461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:45.895622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e1bf-e953-4217-8f47-2587e39f8198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:45.897459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e1bf-e953-4217-8f47-2587e39f8198 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:45.966477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff612-9a57-4504-9018-8742bdaca871 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:45.997303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 532/1040] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:27:45.998396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:45.998679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:45.998953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.005689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.006305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.006650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.006934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.007550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.008963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.015916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 509/1041] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:27:45 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:27:46.016937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.018960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:46.020290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:46.027642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.107921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 533/1042] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:46.136437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a494c4c-2d04-43f9-8f08-2906e574b0f4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.173518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.174144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.174448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.174724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.185305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.186807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:46.202234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:46.203226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:46.204528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:46.204528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:46.204528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:46.204528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 510/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:46.209742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.239031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.314281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5bc12-a3ee-4552-a06e-a148fc50cec4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.349583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.350615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.351090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.351557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.352431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:46.352945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.354481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.371008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 534/1044] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:46.372542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.383707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.436757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.479378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.483643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a2b5-a7a8-40a2-9e42-7f4db5dc0603 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.494432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f17dd-37d0-4eda-981c-e294f1660d88 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.523091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.524276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.524964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.525397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.530485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.531109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.531387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.531658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.534018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.536357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.543302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:46.548183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:46.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:46.551560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:46.552256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:46.553062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:46.553976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:46.556036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 511/1046] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:46.556202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 535/1046] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:46.556346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.556493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.668031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cc96a-337d-42f5-b78a-042f5bbcc7e0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.669965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7209e5-be9a-468e-b914-db31f21b9a27 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.686609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.705736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.706428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.706706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.706992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.707684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:46.708006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.709375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.717226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.718187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.718477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.718755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.723997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 512/1047] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:46.724267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:46.724267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.724267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.727886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.735922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 536/1048] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:46.742471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.753481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:46.836366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e06872-84af-4496-895a-cd946081e9ae None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.852541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b88f9-0678-467a-98b9-2d42087dfd21 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:46.871586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.872242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.872523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.872797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] PATH_INFO: `/v3/users/510ef1ccd4f1494b83c3fe3e282bf1ba` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.887679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] RBAC: Authorizing `identity:get_user(user_id=510ef1ccd4f1494b83c3fe3e282bf1ba)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.888670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:46.889165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.889268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:46.889547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:46.889819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:46.890541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] RBAC: Adding query filter params (role.id=d1c9bf9ea3a7404bb72a7b829bf0733e, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=f1ab09f1996244a2b328a887fe91febd) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:46.896341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:46.896501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 513/1049] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/users/510ef1ccd4f1494b83c3fe3e282bf1ba => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:46.896599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:46.898110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:46.900857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 537/1050] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/role_assignments?user.id=f1ab09f1996244a2b328a887fe91febd&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=d1c9bf9ea3a7404bb72a7b829bf0733e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:46.943863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.017401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b5777-3bc7-42b8-92fe-951f55feea20 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.060334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.060971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.061254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.061530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.134969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.136479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.178342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.247825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 514/1051] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:27:47.249032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.321765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:47.358041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60d8b7-311b-4193-ab8d-d6b98851433c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.396476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03603448-f4ff-44c5-a272-da34368c21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.397116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603448-f4ff-44c5-a272-da34368c21f9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.397394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603448-f4ff-44c5-a272-da34368c21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.397668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03603448-f4ff-44c5-a272-da34368c21f9 None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.398786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03603448-f4ff-44c5-a272-da34368c21f9 None admin] RBAC: Authorizing `identity:get_project(project_id=09ad52cc8ebc449c8d24487c29c1d49d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.400239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03603448-f4ff-44c5-a272-da34368c21f9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.406671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 538/1052] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:47.407926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:47.512414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac3ad5-4bc1-4ea6-beb8-79c4930b33c9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:47.548319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:47.548929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.549243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.550214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.550974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d, user.id=510ef1ccd4f1494b83c3fe3e282bf1ba) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:47.551423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:47.552876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5b2f9-a08e-448c-92a0-5a97e5f45658 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:47.558623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 515/1053] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:47 2022] GET /identity/v3/role_assignments?user.id=510ef1ccd4f1494b83c3fe3e282bf1ba&scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:47.990082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d508-b0c3-4574-8570-ec95d3c4ec2a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.990402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d508-b0c3-4574-8570-ec95d3c4ec2a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.990779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d508-b0c3-4574-8570-ec95d3c4ec2a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:47.992371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 539/1054] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:47.999358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:47.999358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:47.999358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.004720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.018615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:48.019917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:48.093419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 516/1055] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:48.363034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0d508-b0c3-4574-8570-ec95d3c4ec2a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.473659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0d508-b0c3-4574-8570-ec95d3c4ec2a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.507713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.508377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.508659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.508936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] PATH_INFO: `/v3/services/image` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.509520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.510877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:48.520415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:48.521429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:27:48.522383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:48.524286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 540/1056] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 24 16:27:48 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:48.527442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.632140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95454-e384-42b2-b4d4-91ee98a685f8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.664841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.671210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.671828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.672140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.672416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.673053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] RBAC: Adding query filter params (name=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:48.673356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.675524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaec1f9-f44b-48c8-b0fa-41f86420ab46 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:48.685798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 517/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:48.687201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.693366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.800492 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d10ee2-2df7-4ea6-8f90-79447050160b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.836238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.836852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.837133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.837409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.838037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] RBAC: Adding query filter params (type=image) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:48.838341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.839699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:48.844152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75626755-28a4-4cff-83ac-edd0d2d7f3dc None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.844438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75626755-28a4-4cff-83ac-edd0d2d7f3dc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.844714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75626755-28a4-4cff-83ac-edd0d2d7f3dc None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.846298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 518/1058] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:48.846742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 541/1059] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:48.851213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:48.851445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.851703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.852001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.870416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:48.872095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:48.922441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:48.953023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 519/1060] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:48.957545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb449e5-0026-4777-b5fd-e46a124f5a5c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:48.993919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:48.994538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:48.994819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:48.995095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:48.995765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:48.999392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.027274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 542/1061] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:27:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:49.233390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.335763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a86af2-7cba-4bca-9067-05166b8fec27 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:49.362200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:49.377203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632eef3-f284-465a-85e3-ae1f54a96c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:49.377925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632eef3-f284-465a-85e3-ae1f54a96c5b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.378190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632eef3-f284-465a-85e3-ae1f54a96c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.378454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632eef3-f284-465a-85e3-ae1f54a96c5b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.388923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632eef3-f284-465a-85e3-ae1f54a96c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5632eef3-f284-465a-85e3-ae1f54a96c5b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:49.394360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:49.395290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:49.396186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:49.397380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 520/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:49.401700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.476062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45e26f-12ee-4ad3-b9cb-9e8589bee79c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.476347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45e26f-12ee-4ad3-b9cb-9e8589bee79c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.476629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45e26f-12ee-4ad3-b9cb-9e8589bee79c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.478728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 521/1063] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:49.485265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efbee4-109b-4453-85aa-29883403c620 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.485476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efbee4-109b-4453-85aa-29883403c620 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.485759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efbee4-109b-4453-85aa-29883403c620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.509182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d2049-7a64-4a47-a379-150ad3895c79 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:49.511682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-26efbee4-109b-4453-85aa-29883403c620 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:49.512921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-26efbee4-109b-4453-85aa-29883403c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:49.533439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:49.546754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:49.547362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.547642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.547922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.548616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:49.548917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:49.550528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.559355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 543/1064] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:49.565438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.598617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 522/1065] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:49.664364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029f3e8-8b7e-4c73-b85f-b9b3a814b86e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:49.698542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:49.699170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.699449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.699724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.709872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:49.711298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:49.725943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:49.726890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:49.732800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:49.732800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:49.732800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:49.732800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 544/1066] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:49.734653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efbee4-109b-4453-85aa-29883403c620 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.834672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efbee4-109b-4453-85aa-29883403c620 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:49.848399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:49.859925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.873287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:49.874318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:49.874883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:49.875171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:49.875827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:49.876418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:49.879036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:49.898081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 523/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:49.899722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:49.905245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:49.970851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6ba5-f9bf-4b4f-a45a-b0e4a8bdd07c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.007448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.008110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.008392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.008665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.010648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754d945-7150-48e3-98c8-ad0ecb07ab0f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.020180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:50.024922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:50.026037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:50.027926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 545/1068] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:50.034589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.056893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.057895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.058278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.058561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.067738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.069219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:50.081056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:50.082170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:50.083119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 524/1069] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:27:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:50.088228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.122842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.149886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.154749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a5b54-02ea-45a4-92df-74e11dd2afce None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.191101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.191706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.192008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.192285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.192947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:50.193247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.194605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.203055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546999b-7752-4201-9f1c-8ebd5bd6bfea None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.203766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 546/1070] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:50.209923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.241095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.242059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.242496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.242917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.243785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:50.244273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.246042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.265312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 525/1071] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:50.271280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.318213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d276f-4d45-46f0-94eb-559a081bfb49 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.354218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.354880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.355162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.355438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] PATH_INFO: `/v3/users/510ef1ccd4f1494b83c3fe3e282bf1ba` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.356689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] RBAC: Authorizing `identity:get_user(user_id=510ef1ccd4f1494b83c3fe3e282bf1ba)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.358245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.365225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 547/1072] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/users/510ef1ccd4f1494b83c3fe3e282bf1ba => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:50.366195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.371422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.382222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860304a-476c-492d-9b8e-e2c3ac105cda None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.397311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.420187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.420819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.421099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.421376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.432184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.433622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:50.448794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:50.449965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:50.450931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:50.450931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:50.450931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:50.450931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 526/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:50.455873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.481871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e52b6-2cb6-4c61-9ffb-3c78d80fa200 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.518379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.519023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.519305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.519605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.520774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] RBAC: Authorizing `identity:get_project(project_id=09ad52cc8ebc449c8d24487c29c1d49d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.522197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.524029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 548/1074] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:50.529696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.560703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce3db3-5836-4b09-a442-0aeb4f3fd508 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.600623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.601265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.601925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.601925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.602388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:50.602688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.604065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.620093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 527/1075] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:50.622309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.625436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.636392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:50.643824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f4f6-0989-4009-b036-e6ba049a17b0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.681709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.682762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.683274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.683751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d/users/510ef1ccd4f1494b83c3fe3e282bf1ba/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.686170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] RBAC: Authorizing `identity:create_grant(project_id=09ad52cc8ebc449c8d24487c29c1d49d, user_id=510ef1ccd4f1494b83c3fe3e282bf1ba, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.688178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.703961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 549/1076] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:50 2022] PUT /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d/users/510ef1ccd4f1494b83c3fe3e282bf1ba/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:27:50.743470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b9290-cb8c-4060-9d4e-e779795993e8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.783311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.784309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.784756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.785036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.792860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.794330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:50.807250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:50.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:50.809498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:50.812315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 528/1077] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:50.814351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.923687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7521-56ea-4beb-a70d-b25211a76b71 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:50.960545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:50.961156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.961436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.961709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.962398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:50.962701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:50.964144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8656f5-bc67-45aa-844d-17ea4212e66d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:50.976229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 550/1078] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:50.981968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:50.992631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a090e-529b-46b9-8496-913adcc43c95 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:50.992927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a090e-529b-46b9-8496-913adcc43c95 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:50.993185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a090e-529b-46b9-8496-913adcc43c95 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:50.994514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 551/1079] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:51.000732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.001012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.001283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.019661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:51.021548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:51.035963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.062844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.107043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963b5b9-b933-47a6-96bf-ddca017423ca None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.107443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 552/1080] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:51.142671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.143360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.143730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.144033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/f1ab09f1996244a2b328a887fe91febd/roles/d1c9bf9ea3a7404bb72a7b829bf0733e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.146168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=f1ab09f1996244a2b328a887fe91febd, role_id=d1c9bf9ea3a7404bb72a7b829bf0733e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.147591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.163527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 529/1081] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:50 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/f1ab09f1996244a2b328a887fe91febd/roles/d1c9bf9ea3a7404bb72a7b829bf0733e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:51.369507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.472209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048749-eb8d-4ccc-9493-baff02bc1b7a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.506301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.507046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.507248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.507531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.523834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.579712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.581788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.684935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 553/1082] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:51.690806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.797125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec8c0b-0bc1-4294-bafe-4350e3f386bf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:51.822379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.836229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:51.836865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:51.837145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:51.837420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:51.847576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:51.849069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6a4fc551-0015-4794-bc04-f9ce068ff3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:51.864242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:51.865176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:51.866065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:51.866065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:51.866065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:51.868009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 530/1083] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:51.871630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:51.921127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:51.978716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd31b5-1601-4e8e-a907-be237d7dbb06 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:52.016838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:52.017444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.017722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.018626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.019274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:52.019582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:52.020970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194795f-4f08-445a-b31e-ebac5118223f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:52.043990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 554/1084] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:52.631008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c1074-5b76-40ba-84dc-1548522c3287 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.631288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c1074-5b76-40ba-84dc-1548522c3287 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.631569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c1074-5b76-40ba-84dc-1548522c3287 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.632965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 531/1085] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:52.638138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c629d-34ff-4ee6-b426-7b4ce6137219 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:52.638578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c629d-34ff-4ee6-b426-7b4ce6137219 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:52.638807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c629d-34ff-4ee6-b426-7b4ce6137219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:52.657220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-192c629d-34ff-4ee6-b426-7b4ce6137219 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:52.658963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-192c629d-34ff-4ee6-b426-7b4ce6137219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:52.727272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 555/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:53.006689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c1074-5b76-40ba-84dc-1548522c3287 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.070441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1942fe-f7e4-4096-bb75-62e09f3c811d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.070724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1942fe-f7e4-4096-bb75-62e09f3c811d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.071001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1942fe-f7e4-4096-bb75-62e09f3c811d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.072527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 556/1087] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:27:53.077091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.077377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.077652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.095578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:53.097590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:53.114720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c1074-5b76-40ba-84dc-1548522c3287 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.126321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.149973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.150667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.150950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.151225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.162769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:53.165292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 557/1088] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:53.167898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:53.169023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:53.169897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 532/1089] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:53.174860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.274612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c65f-5e79-440d-9100-f6b8320fa41c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.309315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.309916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.310198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.310475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.311133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:53.311439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:53.312822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:53.321164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 558/1090] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:53.326912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.365215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.429853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.432618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd793-3e13-4b89-a319-33c0e355b34d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.459052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.468369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.468997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.469277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.469552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] PATH_INFO: `/v3/users/510ef1ccd4f1494b83c3fe3e282bf1ba` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.470680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] RBAC: Authorizing `identity:get_user(user_id=510ef1ccd4f1494b83c3fe3e282bf1ba)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:53.472200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:53.473933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 533/1091] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/users/510ef1ccd4f1494b83c3fe3e282bf1ba => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:53.479680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.532705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc26aa9-15a5-4350-8ead-71eb5c2eaf3f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.567206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.567833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.568143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.568417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.578548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:53.583272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72db82-6226-4fc6-993b-182433de33cc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:53.584575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:53.585584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:27:53.586958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:53.587622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.587694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 559/1092] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:53.591568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.620040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.620713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.621001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.621280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.680263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.693129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d916-2da7-42ab-9036-0a0710e58c85 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.696153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:53.698382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:53.727061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.727679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.727968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.728252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.728918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:53.729218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:53.730592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:53.740193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 560/1093] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:53.746001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.811044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 534/1094] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:27:53.817585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.819526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.851388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8001212-54f8-4014-bb58-ebecc0f72b6b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.885631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.886249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.886524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.886790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.896864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:53.898294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:53.912654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:53.913624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:53.915106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:53.915106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:53.915106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:53.915106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 561/1095] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:53.919712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:53.922035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f8d9-23db-47b7-89c3-aa1cf1c0e360 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:53.922333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:53.957468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:53.958092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:53.958369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:53.958641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:53.959749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] RBAC: Authorizing `identity:get_project(project_id=09ad52cc8ebc449c8d24487c29c1d49d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:53.961209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:53.962829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 535/1096] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:53.968746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.022558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ba40-fb6a-4fcd-8984-d0e1b2b103fc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.040518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.059982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.060606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.060885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.061211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.061801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:54.062104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.063517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.079998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 562/1097] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:54.080212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e58788-5daa-4d3a-9ef2-71398fa3326c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.082211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.113316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.113908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.114182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.114450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.115168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d, user.id=510ef1ccd4f1494b83c3fe3e282bf1ba) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:54.115476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.116919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.128207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 536/1098] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:53 2022] GET /identity/v3/role_assignments?user.id=510ef1ccd4f1494b83c3fe3e282bf1ba&scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:54.143050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.185913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5acbef-17c5-4d25-9997-ab476e509014 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.223689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.224372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.224652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.224925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.232816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.234279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:27:54.247772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:54.249033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:27:54.249996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 563/1099] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:54.254924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.357428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a78b0-6702-4805-90bd-65ced8bf4a51 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.390661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.391313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.391633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.391910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.392687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:54.393373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.394611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.396372 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.413188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 537/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:54.419274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.517121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5873-723a-4d79-ac41-a1145fa51299 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.538570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.551199 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.551805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.552115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.552392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.618393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.619744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.715439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 564/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:54.721241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.819140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f98e8-3ded-4871-9cc4-6ede0dea8c47 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:54.851722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:54.852369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:54.852650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:54.852922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:54.863087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:54.864554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:54.879079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:54.880430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:27:54.881584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:54.881584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:27:54.881584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:54.881584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 538/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:27:54.885746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:54.909155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:54.982851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1a3d2-bf97-4d86-90d4-cad8c06f7913 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.017157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.017789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.018094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.018464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.019093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:55.019393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.020764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.032097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 565/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:55.037967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:55.085140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:55.139157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff262a1-fb88-4009-9885-f42a41490bd0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.171890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.172533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.172809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.173078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.180159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.181584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:27:55.192640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:55.193693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:27:55.194628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 539/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:27:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:55.199493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:55.282791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:55.303312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82f8b2-5161-4eca-abca-dcb501bbc792 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.336616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.337221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.337504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.337779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.338467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:55.338776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.340229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576b3c9-574e-4dd8-a720-65399f7d9a2d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.352499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 566/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:55.358673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:55.457280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301207f-5f90-4200-a19a-2f175dd8a32a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:55.459432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:55.491041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:55.491642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:55.491924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:55.492358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:55.493091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] RBAC: Adding query filter params (role.id=d1c9bf9ea3a7404bb72a7b829bf0733e, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=f1ab09f1996244a2b328a887fe91febd) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:55.493404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:55.494824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a989e5-18d0-4b5c-90fb-9a1fac509027 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:55.500544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 540/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:55 2022] GET /identity/v3/role_assignments?user.id=f1ab09f1996244a2b328a887fe91febd&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=d1c9bf9ea3a7404bb72a7b829bf0733e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:56.083276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b853727-bb72-4e10-9ed3-838dbe44e071 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:56.083812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b853727-bb72-4e10-9ed3-838dbe44e071 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:56.083812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b853727-bb72-4e10-9ed3-838dbe44e071 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:56.085242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 567/1107] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:56.090995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:56.091470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:56.091574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:56.110421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:56.112135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:56.184127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 541/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:56.451706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b853727-bb72-4e10-9ed3-838dbe44e071 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:56.551093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b853727-bb72-4e10-9ed3-838dbe44e071 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:56.583782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb117c-4f39-4ef2-8983-176ce7db1446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:56.584412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb117c-4f39-4ef2-8983-176ce7db1446 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:56.584687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb117c-4f39-4ef2-8983-176ce7db1446 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:56.584956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb117c-4f39-4ef2-8983-176ce7db1446 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:56.586046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb117c-4f39-4ef2-8983-176ce7db1446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:56.587454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb117c-4f39-4ef2-8983-176ce7db1446 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:56.589037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 568/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:56.594698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:56.687150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:56.691608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c66e5-cc0f-4a1e-8eaa-ffe48a8092fe None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:56.724507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bcb0a-fb41-42b7-b5e1-ab4678a501cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:56.725034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bcb0a-fb41-42b7-b5e1-ab4678a501cd None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:56.725308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bcb0a-fb41-42b7-b5e1-ab4678a501cd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:56.725577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bcb0a-fb41-42b7-b5e1-ab4678a501cd None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:56.726214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bcb0a-fb41-42b7-b5e1-ab4678a501cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:56.727621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bcb0a-fb41-42b7-b5e1-ab4678a501cd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:56.748464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 542/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:56 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:58.676908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf95a-247d-4bed-9570-b685efc1efd6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.677515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf95a-247d-4bed-9570-b685efc1efd6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.677515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf95a-247d-4bed-9570-b685efc1efd6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:58.678804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 569/1111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:58.684172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:58.684560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:58.684725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:58.703503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:58.704711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:58.769185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 543/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:59.037262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf95a-247d-4bed-9570-b685efc1efd6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.050304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.137135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf95a-247d-4bed-9570-b685efc1efd6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.169017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.169627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.169901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.170169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] PATH_INFO: `/v3/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.171530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] RBAC: Authorizing `identity:get_role(role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:59.172900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:59.174719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 570/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/roles/87236176cec04669b7f8d8199ce12228 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:59.180624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.264423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.281052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb5b82-6377-4fd6-9c9b-b5f213a8f9ef None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.315964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.316545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.316822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.317092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] PATH_INFO: `/v3/users/f636d90b5c4e4d9991af474656519364` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.329618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] RBAC: Authorizing `identity:get_user(user_id=f636d90b5c4e4d9991af474656519364)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:59.331516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:59.333139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 544/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/users/f636d90b5c4e4d9991af474656519364 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:59.338876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.419184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.441755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72c6d-07f6-4215-88ab-1451199cc0bb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.475218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.475822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.476141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.476417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.542142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:59.543569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:59.617722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:59.639303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 571/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:59.646173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.743900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed0f4-2d4a-48c0-91e0-d1a61a8e237a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.777500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a70cc-1533-44f5-ab31-43ac64772b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.778123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a70cc-1533-44f5-ab31-43ac64772b56 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.778401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a70cc-1533-44f5-ab31-43ac64772b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.778674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a70cc-1533-44f5-ab31-43ac64772b56 None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.779782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a70cc-1533-44f5-ab31-43ac64772b56 None admin] RBAC: Authorizing `identity:get_project(project_id=09ad52cc8ebc449c8d24487c29c1d49d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:59.781231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a70cc-1533-44f5-ab31-43ac64772b56 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:59.782845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 545/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:59.789426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:59.887201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0453830-5e8b-4dc6-b207-afef78020030 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:59.920537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:59.921128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:59.921407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:59.921680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:59.922394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] RBAC: Adding query filter params (role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d, user.id=f636d90b5c4e4d9991af474656519364) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:59.922699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:59.925106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd88650-dd1f-4afb-b0ab-a171c4dd1748 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:59.930281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 572/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:59 2022] GET /identity/v3/role_assignments?user.id=f636d90b5c4e4d9991af474656519364&scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d&role.id=87236176cec04669b7f8d8199ce12228 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:01.790688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ed9bb-c3e4-4b6e-8253-28c96f2b940c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.791505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ed9bb-c3e4-4b6e-8253-28c96f2b940c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.791505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ed9bb-c3e4-4b6e-8253-28c96f2b940c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.792877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 546/1118] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:01.797817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:01.798217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:01.798364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:01.816483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:01.817649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:01.851693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:01.882807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 573/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:02.154858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ed9bb-c3e4-4b6e-8253-28c96f2b940c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.168163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.254438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ed9bb-c3e4-4b6e-8253-28c96f2b940c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.287712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.288373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.288651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.288925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] PATH_INFO: `/v3/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.290299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] RBAC: Authorizing `identity:get_role(role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.291933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.293264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 547/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/roles/87236176cec04669b7f8d8199ce12228 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.299743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.397845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fc713-2c81-4a3d-a24c-cbec27ae70f5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.431391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.431977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.432263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.432548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] PATH_INFO: `/v3/users/f636d90b5c4e4d9991af474656519364` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.433673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] RBAC: Authorizing `identity:get_user(user_id=f636d90b5c4e4d9991af474656519364)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.435088 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.436699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 574/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/users/f636d90b5c4e4d9991af474656519364 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.442994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.525879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.547400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9befd-9081-49c9-b6cd-26e02a8bc3cf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.581039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d55da9-46bc-4eb4-971b-2e4f154b1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.581587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d55da9-46bc-4eb4-971b-2e4f154b1175 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.581868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d55da9-46bc-4eb4-971b-2e4f154b1175 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.582148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d55da9-46bc-4eb4-971b-2e4f154b1175 None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.583266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d55da9-46bc-4eb4-971b-2e4f154b1175 None admin] RBAC: Authorizing `identity:get_project(project_id=09ad52cc8ebc449c8d24487c29c1d49d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.584730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d55da9-46bc-4eb4-971b-2e4f154b1175 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.586353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 548/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:02 2022] GET /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:02.592352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:02.633617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:02.691592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310e002-7117-4b6f-811e-832d74df5580 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:02.724084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eeaba-509f-4417-b395-92f366eb657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:02.724737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eeaba-509f-4417-b395-92f366eb657b None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:02.725014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eeaba-509f-4417-b395-92f366eb657b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:02.725286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eeaba-509f-4417-b395-92f366eb657b None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d/users/f636d90b5c4e4d9991af474656519364/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:02.727378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eeaba-509f-4417-b395-92f366eb657b None admin] RBAC: Authorizing `identity:create_grant(project_id=09ad52cc8ebc449c8d24487c29c1d49d, user_id=f636d90b5c4e4d9991af474656519364, role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:02.728816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eeaba-509f-4417-b395-92f366eb657b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:02.741452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 575/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:02 2022] PUT /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d/users/f636d90b5c4e4d9991af474656519364/roles/87236176cec04669b7f8d8199ce12228 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:04.601152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b667976-8b18-4608-80ec-1d34001ff552 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.601729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b667976-8b18-4608-80ec-1d34001ff552 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.601729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b667976-8b18-4608-80ec-1d34001ff552 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.603089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 549/1124] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:04.608351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:04.608752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:04.608902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:04.628238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:04.629421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:04.694439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 576/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:04.963779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b667976-8b18-4608-80ec-1d34001ff552 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.064190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b667976-8b18-4608-80ec-1d34001ff552 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.098499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.099120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.099400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.099672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] PATH_INFO: `/v3/roles/87236176cec04669b7f8d8199ce12228` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.101182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] RBAC: Authorizing `identity:get_role(role_id=87236176cec04669b7f8d8199ce12228)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.102553 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.103876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 550/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:04 2022] GET /identity/v3/roles/87236176cec04669b7f8d8199ce12228 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:05.109599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.140108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.210448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776e8e9-7260-4edd-897a-6723305c5d96 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.243620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.244210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.244494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.244773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] PATH_INFO: `/v3/users/f636d90b5c4e4d9991af474656519364` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.245884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] RBAC: Authorizing `identity:get_user(user_id=f636d90b5c4e4d9991af474656519364)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.247310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.248912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 577/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/users/f636d90b5c4e4d9991af474656519364 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:05.254688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.270516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.351794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9cf64-14e1-49bc-850a-21a2e4ab50ff None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.384227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.384816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.385089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.385356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.450689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.452152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.465726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.546168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 551/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:05.552190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.647475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.651559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b76ad-5a7e-4394-a045-9e44a345ecd9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.686759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626998-62f2-4a95-b53e-6115da171f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.687397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626998-62f2-4a95-b53e-6115da171f61 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.687677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626998-62f2-4a95-b53e-6115da171f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.687974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626998-62f2-4a95-b53e-6115da171f61 None admin] PATH_INFO: `/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.689626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45626998-62f2-4a95-b53e-6115da171f61 None admin] RBAC: Authorizing `identity:get_project(project_id=09ad52cc8ebc449c8d24487c29c1d49d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.691058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45626998-62f2-4a95-b53e-6115da171f61 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.692742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 578/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/projects/09ad52cc8ebc449c8d24487c29c1d49d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:05.698901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:05.795054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da2554-351c-42d4-9f22-1e2f36730ffa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:05.809227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:05.829378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:05.829999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:05.830364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:05.830644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:05.831365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] RBAC: Adding query filter params (role.id=87236176cec04669b7f8d8199ce12228, scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d, user.id=f636d90b5c4e4d9991af474656519364) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:05.831677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:05.833129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbfbdc-259e-4786-bd9f-90502909dcd2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:05.838688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 552/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:05 2022] GET /identity/v3/role_assignments?user.id=f636d90b5c4e4d9991af474656519364&scope.project.id=09ad52cc8ebc449c8d24487c29c1d49d&role.id=87236176cec04669b7f8d8199ce12228 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:07.774709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975c16e-b267-493d-8bca-93748818d6b4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.775270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975c16e-b267-493d-8bca-93748818d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.775270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975c16e-b267-493d-8bca-93748818d6b4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.776623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 579/1131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:07.781786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:07.782219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:07.782373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:07.800510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:07.801696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:07.866126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 553/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:08.146789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975c16e-b267-493d-8bca-93748818d6b4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.245645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975c16e-b267-493d-8bca-93748818d6b4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.279183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09897733-e1f3-4942-b47e-8a0751e9ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.279792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09897733-e1f3-4942-b47e-8a0751e9ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.280229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09897733-e1f3-4942-b47e-8a0751e9ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.280507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09897733-e1f3-4942-b47e-8a0751e9ccc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.281594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09897733-e1f3-4942-b47e-8a0751e9ccc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.283650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09897733-e1f3-4942-b47e-8a0751e9ccc4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.285242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 580/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:08.291374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:08.387441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fb966-849f-4a5e-afa4-fa9ddcc116e3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:08.397891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:08.422626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876e80e-752d-4ea9-b3fd-aeb11faef391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:08.423227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876e80e-752d-4ea9-b3fd-aeb11faef391 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:08.423656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876e80e-752d-4ea9-b3fd-aeb11faef391 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:08.424047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876e80e-752d-4ea9-b3fd-aeb11faef391 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:08.424704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876e80e-752d-4ea9-b3fd-aeb11faef391 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:08.426131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876e80e-752d-4ea9-b3fd-aeb11faef391 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:08.451782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 554/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:28:08 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:28:10.446738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f2b40-a637-49d0-bea9-f8368c328b0e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.447289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f2b40-a637-49d0-bea9-f8368c328b0e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.447289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f2b40-a637-49d0-bea9-f8368c328b0e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.448642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 581/1135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:10.453752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.454162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.454302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.474226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:10.475403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:10.541474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 555/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:10.810630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f2b40-a637-49d0-bea9-f8368c328b0e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:10.845266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:10.909450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f2b40-a637-49d0-bea9-f8368c328b0e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:10.942024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98925f-7592-4407-86b6-2d58705e24d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:10.942651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98925f-7592-4407-86b6-2d58705e24d5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:10.942931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98925f-7592-4407-86b6-2d58705e24d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:10.943203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98925f-7592-4407-86b6-2d58705e24d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:10.944326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98925f-7592-4407-86b6-2d58705e24d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:10.945761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98925f-7592-4407-86b6-2d58705e24d5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:10.947324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 582/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:10.953008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:11.041795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:11.052776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b802a5-64fb-497d-912e-03b25ed646ce None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:11.086317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831931bc-52c6-4be7-beda-47fe3329d238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:11.086859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831931bc-52c6-4be7-beda-47fe3329d238 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:11.087141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831931bc-52c6-4be7-beda-47fe3329d238 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:11.087411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831931bc-52c6-4be7-beda-47fe3329d238 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:11.088093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831931bc-52c6-4be7-beda-47fe3329d238 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:11.089519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831931bc-52c6-4be7-beda-47fe3329d238 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:11.109640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 556/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:10 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:28:13.035462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209e3b-5d02-4e06-adaf-69091d0d5725 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.035866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209e3b-5d02-4e06-adaf-69091d0d5725 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.036032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209e3b-5d02-4e06-adaf-69091d0d5725 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.037380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 583/1139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:13.042799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.043235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.043344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.061815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:13.063023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:13.127520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 557/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:28:13.396753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209e3b-5d02-4e06-adaf-69091d0d5725 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.494135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209e3b-5d02-4e06-adaf-69091d0d5725 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.495604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.527776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.528414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.528690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.528958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.539185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:13.544075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:13.545258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:28:13.546514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 584/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:13.552301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.627867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:13.651938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b24613-1687-40d8-bb61-7f1d5e4f1709 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.686241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.686846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.687126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.687398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.688089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:13.688393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.689953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.697768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 558/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.703558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.806069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982da3c-cf04-4dbe-b92a-d09cd7c41923 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.838201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.838817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.839092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.839363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] PATH_INFO: `/v3/users/f842f0468e4844568fb68d39e3561a14` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:13.852538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] RBAC: Authorizing `identity:get_user(user_id=f842f0468e4844568fb68d39e3561a14)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:13.853955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:13.855533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 585/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/users/f842f0468e4844568fb68d39e3561a14 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:13.862484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:13.958238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de078728-12e6-46e4-bcc0-fe043fbde580 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:13.990797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:13.991386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:13.991665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:13.991965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:14.000110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:14.058512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:14.059966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:14.154691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 559/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:14.161365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:14.182384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:14.257240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400badf-7b46-43c2-8246-fb5a60b405ce None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:14.291894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dffc77-360d-48a5-b89a-26bc2a2794b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:14.292541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dffc77-360d-48a5-b89a-26bc2a2794b2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:14.292818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dffc77-360d-48a5-b89a-26bc2a2794b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:14.293090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dffc77-360d-48a5-b89a-26bc2a2794b2 None admin] PATH_INFO: `/v3/projects/18baf9b02bea44deae96ef23d9050394` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:14.294182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dffc77-360d-48a5-b89a-26bc2a2794b2 None admin] RBAC: Authorizing `identity:get_project(project_id=18baf9b02bea44deae96ef23d9050394)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:14.295591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dffc77-360d-48a5-b89a-26bc2a2794b2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:14.297225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 586/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:14 2022] GET /identity/v3/projects/18baf9b02bea44deae96ef23d9050394 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:14.303498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:14.341884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:14.400898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c118aa-39cb-4991-aebe-f776f4dd38ad None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:14.433102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:14.433687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:14.433961 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:14.434229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:14.434938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=18baf9b02bea44deae96ef23d9050394, user.id=f842f0468e4844568fb68d39e3561a14) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:14.435240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:14.436662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18928f-4977-4810-a5df-7165ee879431 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:14.442462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 560/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:14 2022] GET /identity/v3/role_assignments?user.id=f842f0468e4844568fb68d39e3561a14&scope.project.id=18baf9b02bea44deae96ef23d9050394&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:16.380445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fee1c-71d6-4403-b102-f0a1dd5506b6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:16.380984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fee1c-71d6-4403-b102-f0a1dd5506b6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:16.380984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fee1c-71d6-4403-b102-f0a1dd5506b6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:16.382347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 587/1147] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:16.387524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:16.387980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:16.387980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:16.406242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:16.408174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:16.473038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 561/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:16.741149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fee1c-71d6-4403-b102-f0a1dd5506b6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:16.827783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:16.841452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fee1c-71d6-4403-b102-f0a1dd5506b6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:16.875205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:16.875819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:16.876652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:16.877370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:16.887087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:16.891699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:16.893367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:28:16.894552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 588/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:16.899901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:16.930341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:16.998983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7822-7e09-49a4-830f-307b4162fa58 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:17.033701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:17.034308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.034582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.034855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.035518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:17.035820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:17.037216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:17.045505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 562/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:17.051807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:17.150810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a66a5-b03f-4e5a-86e3-7a6be824de31 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:17.184186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:17.184809 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.185092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.185368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] PATH_INFO: `/v3/users/f842f0468e4844568fb68d39e3561a14` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.186492 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] RBAC: Authorizing `identity:get_user(user_id=f842f0468e4844568fb68d39e3561a14)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:17.187915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:17.190129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 589/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:17 2022] GET /identity/v3/users/f842f0468e4844568fb68d39e3561a14 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:17.195956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:17.289114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:17.296122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e59154-3e7a-488f-94ca-438a57f46ac0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:17.330254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5107a4b-5e07-4407-83aa-fad862de7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:17.330895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5107a4b-5e07-4407-83aa-fad862de7002 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.331173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5107a4b-5e07-4407-83aa-fad862de7002 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.331442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5107a4b-5e07-4407-83aa-fad862de7002 None admin] PATH_INFO: `/v3/projects/18baf9b02bea44deae96ef23d9050394` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.332684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5107a4b-5e07-4407-83aa-fad862de7002 None admin] RBAC: Authorizing `identity:get_project(project_id=18baf9b02bea44deae96ef23d9050394)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:17.334107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5107a4b-5e07-4407-83aa-fad862de7002 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:17.335730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 563/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:17 2022] GET /identity/v3/projects/18baf9b02bea44deae96ef23d9050394 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:17.341692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:17.361128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:17.442094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249eac9-d775-4106-9ec1-7aa05088ffa1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:17.474735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e0c66-1645-4dfa-8209-e2cf1038decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:17.475384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e0c66-1645-4dfa-8209-e2cf1038decf None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:17.475666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e0c66-1645-4dfa-8209-e2cf1038decf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:17.475966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e0c66-1645-4dfa-8209-e2cf1038decf None admin] PATH_INFO: `/v3/projects/18baf9b02bea44deae96ef23d9050394/users/f842f0468e4844568fb68d39e3561a14/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:17.477852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e0c66-1645-4dfa-8209-e2cf1038decf None admin] RBAC: Authorizing `identity:create_grant(project_id=18baf9b02bea44deae96ef23d9050394, user_id=f842f0468e4844568fb68d39e3561a14, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:17.479270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e0c66-1645-4dfa-8209-e2cf1038decf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:17.491715 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 590/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:17 2022] PUT /identity/v3/projects/18baf9b02bea44deae96ef23d9050394/users/f842f0468e4844568fb68d39e3561a14/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:19.354609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47aa2b-3f2b-43c9-9eef-5f2e2d80c964 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.355189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47aa2b-3f2b-43c9-9eef-5f2e2d80c964 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.355189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47aa2b-3f2b-43c9-9eef-5f2e2d80c964 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.356550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 564/1154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:19.362235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.362612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.362784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.380554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:19.381731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:19.446888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 591/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:19.738416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47aa2b-3f2b-43c9-9eef-5f2e2d80c964 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.843146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47aa2b-3f2b-43c9-9eef-5f2e2d80c964 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:19.878229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:19.878841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:19.879117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:19.879386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:19.889546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:19.894936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:19.896065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:28:19.897325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 565/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:19.902024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:19.910048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:19.998025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe96627-9fd6-4a68-881a-d4e157016128 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.030925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.031522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.031796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.032100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.033024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:20.033324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.034678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.042812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 592/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:20.049486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.074969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.149107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fabda-7279-4cfd-954b-58846afa9acc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.182091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.182710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.182992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.183267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] PATH_INFO: `/v3/users/f842f0468e4844568fb68d39e3561a14` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.184412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] RBAC: Authorizing `identity:get_user(user_id=f842f0468e4844568fb68d39e3561a14)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.185845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.187415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 566/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:20 2022] GET /identity/v3/users/f842f0468e4844568fb68d39e3561a14 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:20.193883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.261791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.293659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a265c-358e-49d3-acf3-9884d8bb9a78 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.326087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.326677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.326953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.327222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.393053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.394471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.457545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.491645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 593/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:20.497539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.585858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:20.596131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921598c-00e9-4f5c-84f2-1efac78324df None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.628429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6b96a-168c-4e82-9fa9-06d12218fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.629038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6b96a-168c-4e82-9fa9-06d12218fa1c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.629314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6b96a-168c-4e82-9fa9-06d12218fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.629580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6b96a-168c-4e82-9fa9-06d12218fa1c None admin] PATH_INFO: `/v3/projects/18baf9b02bea44deae96ef23d9050394` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.630674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6b96a-168c-4e82-9fa9-06d12218fa1c None admin] RBAC: Authorizing `identity:get_project(project_id=18baf9b02bea44deae96ef23d9050394)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.632112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6b96a-168c-4e82-9fa9-06d12218fa1c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.633701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 567/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:20 2022] GET /identity/v3/projects/18baf9b02bea44deae96ef23d9050394 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:20.639698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:20.735902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987b60d-e4f6-4f27-8767-bb81853d42bc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:20.768966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:20.769553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:20.769832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:20.770104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:20.770817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=18baf9b02bea44deae96ef23d9050394, user.id=f842f0468e4844568fb68d39e3561a14) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:20.771123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:20.772558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1710f-ae87-4c0e-9271-77e4fcc7b059 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:20.778079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 594/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:20 2022] GET /identity/v3/role_assignments?user.id=f842f0468e4844568fb68d39e3561a14&scope.project.id=18baf9b02bea44deae96ef23d9050394&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:22.687534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3e7db-2ddc-49bc-81d8-0de8068b4844 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:22.688099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3e7db-2ddc-49bc-81d8-0de8068b4844 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:22.688099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3e7db-2ddc-49bc-81d8-0de8068b4844 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:22.690133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 568/1162] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:22.695341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:22.695776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:22.695876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:22.714237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:22.715417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:22.740582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:22.781207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 595/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:28:23.047145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3e7db-2ddc-49bc-81d8-0de8068b4844 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.147025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3e7db-2ddc-49bc-81d8-0de8068b4844 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.179620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.180240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.180521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.180796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.207040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.246856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.248473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.342478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 569/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:23.349107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.446061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d6cd3-dbfc-4e57-9d90-b613603d855b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.478963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.479578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.479851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.480146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.486617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.488057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-fdb2bc0e-fb21-4198-ad1b-e5404bf32000 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:23.498933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:23.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:28:23.501980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 596/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:23.506559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:23.562988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:23.604048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ba7b6-1533-4b1a-ab8d-0fb92e38b6ab None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:23.637088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:23.637677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:23.637952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:23.638222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:23.638854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:23.639151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:23.640542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc60bfd-60bb-45ad-9951-43a2ff70071a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:23.648415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 570/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 16:28:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:25.544239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f86d5-3068-4197-95f5-90454dff280b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.544800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f86d5-3068-4197-95f5-90454dff280b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.544982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f86d5-3068-4197-95f5-90454dff280b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.546922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 597/1167] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:28:25.553023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed0a1-ff5a-4ce4-9f48-132223218e5b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:25.553486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed0a1-ff5a-4ce4-9f48-132223218e5b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:25.553594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed0a1-ff5a-4ce4-9f48-132223218e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:25.572999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-860ed0a1-ff5a-4ce4-9f48-132223218e5b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:25.574204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-860ed0a1-ff5a-4ce4-9f48-132223218e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:25.650481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 571/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:28:25.948209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f86d5-3068-4197-95f5-90454dff280b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:25.974247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:26.050704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f86d5-3068-4197-95f5-90454dff280b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:26.084138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f2517-5d09-4210-a1ea-3d0bc9df4001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:26.084736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f2517-5d09-4210-a1ea-3d0bc9df4001 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:26.085009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f2517-5d09-4210-a1ea-3d0bc9df4001 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:26.085279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f2517-5d09-4210-a1ea-3d0bc9df4001 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:26.085938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f2517-5d09-4210-a1ea-3d0bc9df4001 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:26.087275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f2517-5d09-4210-a1ea-3d0bc9df4001 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:26.102568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 598/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Aug 24 16:28:25 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:28:28.020266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aa981-6e89-45ea-874a-ec490b0159de None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.020992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aa981-6e89-45ea-874a-ec490b0159de None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.020992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aa981-6e89-45ea-874a-ec490b0159de None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.023033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 572/1170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:28.028513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.028945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.029074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.048716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:28.049718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:28.116224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 599/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:28.430451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aa981-6e89-45ea-874a-ec490b0159de None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.483241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.531420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aa981-6e89-45ea-874a-ec490b0159de None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.566220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396726f7-fe9c-42a0-8297-bc3982d1ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.566838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396726f7-fe9c-42a0-8297-bc3982d1ec6d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.567116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396726f7-fe9c-42a0-8297-bc3982d1ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.567389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396726f7-fe9c-42a0-8297-bc3982d1ec6d None admin] PATH_INFO: `/v3/domains/596a40b7f9bb407a87579294eb8918e8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.577379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396726f7-fe9c-42a0-8297-bc3982d1ec6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=596a40b7f9bb407a87579294eb8918e8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.579046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396726f7-fe9c-42a0-8297-bc3982d1ec6d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.580971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 573/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Aug 24 16:28:28 2022] GET /identity/v3/domains/596a40b7f9bb407a87579294eb8918e8 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:28.586775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:28.600597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:28.687150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e62ca-ffd3-4441-9b78-2eb748a536f0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:28.722059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f83984-7a2f-446c-bcb8-91f310c9f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:28.722660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f83984-7a2f-446c-bcb8-91f310c9f741 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:28.722937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f83984-7a2f-446c-bcb8-91f310c9f741 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:28.723211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f83984-7a2f-446c-bcb8-91f310c9f741 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:28.723850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f83984-7a2f-446c-bcb8-91f310c9f741 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:28.725383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f83984-7a2f-446c-bcb8-91f310c9f741 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:28.750411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 600/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:28:28 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:30.664989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a1cc5-22f0-460a-9ec8-f97026b50854 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:30.665795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a1cc5-22f0-460a-9ec8-f97026b50854 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:30.665795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a1cc5-22f0-460a-9ec8-f97026b50854 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:30.667127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 574/1174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:30.672333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:30.673451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:30.673451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:30.691433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:30.692638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:30.748850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:30.758552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 601/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:31.025160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a1cc5-22f0-460a-9ec8-f97026b50854 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:31.125410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a1cc5-22f0-460a-9ec8-f97026b50854 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:31.137627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:31.158992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d4aa2-8812-4e7a-a1f0-0cac7b0738f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:31.159610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d4aa2-8812-4e7a-a1f0-0cac7b0738f3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.159892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d4aa2-8812-4e7a-a1f0-0cac7b0738f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.160196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d4aa2-8812-4e7a-a1f0-0cac7b0738f3 None admin] PATH_INFO: `/v3/domains/596a40b7f9bb407a87579294eb8918e8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:31.161305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d4aa2-8812-4e7a-a1f0-0cac7b0738f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=596a40b7f9bb407a87579294eb8918e8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:31.162725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d4aa2-8812-4e7a-a1f0-0cac7b0738f3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:31.164323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 575/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Aug 24 16:28:31 2022] GET /identity/v3/domains/596a40b7f9bb407a87579294eb8918e8 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:31.170253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:31.264892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b857c1-0223-4bb7-9465-f2feef09a46e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:31.299260 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90266a-6ad3-4b00-9845-1b4d457ba470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:31.300006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90266a-6ad3-4b00-9845-1b4d457ba470 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:31.300288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90266a-6ad3-4b00-9845-1b4d457ba470 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:31.300561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90266a-6ad3-4b00-9845-1b4d457ba470 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:31.301206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90266a-6ad3-4b00-9845-1b4d457ba470 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:31.302613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90266a-6ad3-4b00-9845-1b4d457ba470 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:31.323084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 602/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:31 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:28:33.258625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d026-f810-4691-9d59-68d699b0a0f8 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.259169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d026-f810-4691-9d59-68d699b0a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.259169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d026-f810-4691-9d59-68d699b0a0f8 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.260575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 576/1178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:33.265910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.266953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.267288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.284848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:33.285964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:33.337125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:33.358848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 603/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:33.631927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3d026-f810-4691-9d59-68d699b0a0f8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.730926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3d026-f810-4691-9d59-68d699b0a0f8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.763867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.764512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.764792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.765067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.775846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:33.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:33.782301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:33.784204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:28:33.785245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 577/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:33.789840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.887476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af65ccc-2301-480c-ad57-6ced5a926e73 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:33.921927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:33.922582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:33.922947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:33.923230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:33.923888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:33.924223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:33.925589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:33.933418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 604/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:33.939849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:33.981026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.038332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533ea99-9da6-4889-b6d6-ecb5fc8c86d0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.071240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.071863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.072183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.072459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] PATH_INFO: `/v3/users/38a571da9a234c6d9531debc8df83c7f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.085674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] RBAC: Authorizing `identity:get_user(user_id=38a571da9a234c6d9531debc8df83c7f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.087104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.088735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 578/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:33 2022] GET /identity/v3/users/38a571da9a234c6d9531debc8df83c7f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:28:34.094989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.137114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.193852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2470e-3529-4865-b5e6-5485404f4cc9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.226233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.226829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.227109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.227386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.292497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.293921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.334499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.390268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 605/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:34.396766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.490732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:34.494667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4808e3e-fc8d-4e63-912d-368cc9133a23 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.529549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb4fe7-0de1-43e7-91f2-f2b40fe5d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.530162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb4fe7-0de1-43e7-91f2-f2b40fe5d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.530438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb4fe7-0de1-43e7-91f2-f2b40fe5d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.530708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb4fe7-0de1-43e7-91f2-f2b40fe5d6f4 None admin] PATH_INFO: `/v3/projects/47418b918aa04644b9d17500bc4a2f50` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.532310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb4fe7-0de1-43e7-91f2-f2b40fe5d6f4 None admin] RBAC: Authorizing `identity:get_project(project_id=47418b918aa04644b9d17500bc4a2f50)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.533752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb4fe7-0de1-43e7-91f2-f2b40fe5d6f4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.535355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 579/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/projects/47418b918aa04644b9d17500bc4a2f50 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:34.541702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:34.640501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb01c-a374-468a-9bd6-a357ea1f3112 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:34.677660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:34.678257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:34.678534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:34.678803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:34.679520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=47418b918aa04644b9d17500bc4a2f50, user.id=38a571da9a234c6d9531debc8df83c7f) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:34.679822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:34.681256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f6eb2-a8cd-4ae0-a6bf-1215bdbcb98a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:34.686387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 606/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:34 2022] GET /identity/v3/role_assignments?user.id=38a571da9a234c6d9531debc8df83c7f&scope.project.id=47418b918aa04644b9d17500bc4a2f50&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:36.551213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770607-ecce-42a7-8c3c-3e10c1ecaf54 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.551773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770607-ecce-42a7-8c3c-3e10c1ecaf54 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.551773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770607-ecce-42a7-8c3c-3e10c1ecaf54 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.553125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 580/1186] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:36.558397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:36.558771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:36.558957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:36.572364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:36.577892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:36.579277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:36.644932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 607/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:36.913442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81770607-ecce-42a7-8c3c-3e10c1ecaf54 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.012035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81770607-ecce-42a7-8c3c-3e10c1ecaf54 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.045374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.045984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.046257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.046525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.056923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:37.061492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:37.062614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:28:37.064072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 581/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:37.068969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.163280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be32dd-868f-441e-8977-309dd265a4d9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.196624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.197168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.197450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.197729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.198388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:37.198693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.200086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.202185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.208899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 608/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:37.214721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.243575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.314774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abfbee-2279-4ff4-a697-c1aa484d8bcf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.347796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.348430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.348706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.348971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] PATH_INFO: `/v3/users/38a571da9a234c6d9531debc8df83c7f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.350073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] RBAC: Authorizing `identity:get_user(user_id=38a571da9a234c6d9531debc8df83c7f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.351482 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.353063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 582/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/users/38a571da9a234c6d9531debc8df83c7f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:37.358806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.457434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c079f-c255-4796-a03b-7111b49e09bf None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.490696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8c877-c32a-4a19-9c23-438b1c728b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.491314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8c877-c32a-4a19-9c23-438b1c728b20 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.491590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8c877-c32a-4a19-9c23-438b1c728b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.491862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8c877-c32a-4a19-9c23-438b1c728b20 None admin] PATH_INFO: `/v3/projects/47418b918aa04644b9d17500bc4a2f50` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.492977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8c877-c32a-4a19-9c23-438b1c728b20 None admin] RBAC: Authorizing `identity:get_project(project_id=47418b918aa04644b9d17500bc4a2f50)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.494392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8c877-c32a-4a19-9c23-438b1c728b20 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.496023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 609/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:37 2022] GET /identity/v3/projects/47418b918aa04644b9d17500bc4a2f50 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:37.502368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:37.591743 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:37.599324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac674f-7aaf-452b-83b3-f228e88ee7aa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:37.632319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9c8a-fc00-4bec-b4c5-dfbbe865361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:37.632959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9c8a-fc00-4bec-b4c5-dfbbe865361f None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:37.633235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9c8a-fc00-4bec-b4c5-dfbbe865361f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:37.633505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9c8a-fc00-4bec-b4c5-dfbbe865361f None admin] PATH_INFO: `/v3/projects/47418b918aa04644b9d17500bc4a2f50/users/38a571da9a234c6d9531debc8df83c7f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:37.635357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a9c8a-fc00-4bec-b4c5-dfbbe865361f None admin] RBAC: Authorizing `identity:create_grant(project_id=47418b918aa04644b9d17500bc4a2f50, user_id=38a571da9a234c6d9531debc8df83c7f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:37.636799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a9c8a-fc00-4bec-b4c5-dfbbe865361f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:37.649407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 583/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:37 2022] PUT /identity/v3/projects/47418b918aa04644b9d17500bc4a2f50/users/38a571da9a234c6d9531debc8df83c7f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:28:39.496955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7afdee-c02e-4e57-8cc3-75777761ccd0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.497508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7afdee-c02e-4e57-8cc3-75777761ccd0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.497508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7afdee-c02e-4e57-8cc3-75777761ccd0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.498835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 610/1193] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:39.504622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:39.505004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:39.505182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:39.523277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:39.524503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:39.589349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 584/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:39.862439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7afdee-c02e-4e57-8cc3-75777761ccd0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:39.929559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:39.965207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7afdee-c02e-4e57-8cc3-75777761ccd0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.000652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.001585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.001866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.002139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.012864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:40.018519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:40.019722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:28:40.020608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 611/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:28:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:40.028480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.112751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.130051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee6303-26e1-4c94-a200-dda19b065174 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.163708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.164343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.164621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.164891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.165797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:40.166098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.167454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.175489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 585/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:40.181312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.285386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a81e55-68c6-46a0-ae4e-3707b115ba74 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.314439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.323145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.323769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.324081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.324352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] PATH_INFO: `/v3/users/38a571da9a234c6d9531debc8df83c7f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.325456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] RBAC: Authorizing `identity:get_user(user_id=38a571da9a234c6d9531debc8df83c7f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.326866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.328496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 612/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/users/38a571da9a234c6d9531debc8df83c7f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:40.334445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.432534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b04b-e2df-40d7-9066-f3d670c5ab4d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.464727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.465323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.465602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.465877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.480289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.531487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.533020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.628877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 586/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:40.634833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.740035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707074a-746c-4a69-8686-61c6043c9f0d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.773616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590ad64-8639-4f9f-97e8-d33e99d973e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.774242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590ad64-8639-4f9f-97e8-d33e99d973e8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.774520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590ad64-8639-4f9f-97e8-d33e99d973e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.774791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590ad64-8639-4f9f-97e8-d33e99d973e8 None admin] PATH_INFO: `/v3/projects/47418b918aa04644b9d17500bc4a2f50` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.775888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590ad64-8639-4f9f-97e8-d33e99d973e8 None admin] RBAC: Authorizing `identity:get_project(project_id=47418b918aa04644b9d17500bc4a2f50)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.777320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590ad64-8639-4f9f-97e8-d33e99d973e8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.778928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 613/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/projects/47418b918aa04644b9d17500bc4a2f50 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:40.785266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:40.831345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:40.882875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef134e-857a-43ee-8463-18ecdc001a4d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:40.916003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:40.916599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:40.916876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:40.917148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:40.917870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=47418b918aa04644b9d17500bc4a2f50, user.id=38a571da9a234c6d9531debc8df83c7f) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:40.918179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:40.919594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9edfd-d811-4625-a23d-ee186367edd1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:40.925266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 587/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:40 2022] GET /identity/v3/role_assignments?user.id=38a571da9a234c6d9531debc8df83c7f&scope.project.id=47418b918aa04644b9d17500bc4a2f50&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:50.769897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=69 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:50.922773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=45 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:36.992801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9090577-208f-45b4-98a7-2e39df26c934 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:36.993435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9090577-208f-45b4-98a7-2e39df26c934 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:36.993435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9090577-208f-45b4-98a7-2e39df26c934 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:36.994722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 614/1201] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:37.006026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.006026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.006026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.031067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:37.032573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:37.132716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 588/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:37.440850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9090577-208f-45b4-98a7-2e39df26c934 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.582536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9090577-208f-45b4-98a7-2e39df26c934 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.628884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.629536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.629834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.630143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.643567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.645226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:37.663156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:37.664569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:37.665542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:37.665542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:37.665542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:37.665542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 615/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:37.671894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.805266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1579e70-a49d-416b-b85c-0b5e34fa1f5f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:37.844724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:37.851336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:37.852054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:37.852336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:37.852652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:37.853550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:37.853947 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:37.855760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bb9d3-f227-40d5-aadb-d8231b57fa4d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:37.870178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 589/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:37.877162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:37.960615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:38.031009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a52f5-4231-4541-b6b1-f19d8a9877c3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:38.071721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706d8f0-274f-4472-af92-c02b1575566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:38.072503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706d8f0-274f-4472-af92-c02b1575566f None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:38.072816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706d8f0-274f-4472-af92-c02b1575566f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:38.073095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706d8f0-274f-4472-af92-c02b1575566f None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:38.073769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706d8f0-274f-4472-af92-c02b1575566f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:38.075186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706d8f0-274f-4472-af92-c02b1575566f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:38.111981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 616/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 16:29:37 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:40.294930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128a44a-fb75-475f-ba44-d656f47840b1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.294930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128a44a-fb75-475f-ba44-d656f47840b1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.294930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128a44a-fb75-475f-ba44-d656f47840b1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.296063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 590/1206] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:40.300792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.301237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.301330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.321954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:40.323216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:40.404982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 617/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:40.674660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128a44a-fb75-475f-ba44-d656f47840b1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.801895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128a44a-fb75-475f-ba44-d656f47840b1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:40.840327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:40.841034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:40.841358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:40.841654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:40.853260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:40.858677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:40.860235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:40.861305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 591/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:40.865959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:40.908556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:40.980416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d8aad-d956-4925-ac14-0237300f0a95 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.017976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.018592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.018879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.019218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.019837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:41.020163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.021541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.031188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 618/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:29:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:41.037228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.097599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.155510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27894af-0377-493e-97e6-31335a012ab0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.192628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.193304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.193625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.193904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.206225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.207694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:41.222524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:41.224133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:41.225054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:41.225054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:41.225054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:41.225054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 592/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:41.230180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.354495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.358978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb46d2-c336-460c-8ad4-3c672cda11d9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.405655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.406461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.406984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.407304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.408290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:41.409109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.411244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.431254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 619/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:41.437863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.554732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8098c-ce3a-4773-b936-e365d3cac90e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.556507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:41.593315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.594034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.594317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.594592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.602735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.604250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:41.617128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:41.618447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:41.619638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 593/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:41.624906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.748094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df47790-d6e6-4813-9eb9-64ad3189c461 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.790646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.791717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.791977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.792280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:41.793195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:41.793516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:41.795440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:41.816396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 620/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:41.822604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:41.951156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336134-523c-4c62-aa27-c74e72d8c3a6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:41.992169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:41.992850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:41.993160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:41.993437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:42.037036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:42.084952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:42.086563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:42.213889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 594/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:29:42.221106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:42.243873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:42.360696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bb621-a2ed-4dac-bb51-94e1aad81ac5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:42.406969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:42.407898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:42.408339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:42.408818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:42.426473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:42.428103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:42.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:42.451912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:42.453748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:42.453748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:42.453748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:42.456910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 621/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:42.459766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:42.524552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:42.569717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a5ab1-f2a5-4de7-8477-e07c24179a59 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:42.607375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:42.608302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:42.608719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:42.609132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:42.609981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:42.610474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:42.612099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:42.629201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 595/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:42.635996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:42.737475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:42.772581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b510fd-fbd8-43a7-8e1a-7463fe5a344d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:42.809864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:42.810583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:42.811008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:42.811392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:42.818295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:42.819848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:42.833180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:42.834406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:42.835549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:42.837115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 622/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:42.844201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:42.956835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884eb4-2f21-4172-aa92-4ee31dee57eb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:42.983070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:42.998670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:42.999330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:42.999616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:42.999910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.002632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:43.002632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:43.002632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd8746-6e94-4dcc-a5f8-f2c94b1a8a91 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:43.016734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 596/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:43.022571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:43.147017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ea825-abb6-4919-aae1-7f78a93c0e65 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:43.180256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:43.188244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:43.189130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:43.189611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:43.190179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:43.191309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=e605979196e14110a19a6aff30f2006d) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:43.191757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:43.193584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9ee2e-7b44-4799-a983-29ddc337909f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:43.199416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 623/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:43 2022] GET /identity/v3/role_assignments?user.id=e605979196e14110a19a6aff30f2006d&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:45.189265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632165d-3c6e-4f54-b2eb-dd2c3b542593 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.189858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632165d-3c6e-4f54-b2eb-dd2c3b542593 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.189858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632165d-3c6e-4f54-b2eb-dd2c3b542593 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.191329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 597/1220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:45.196679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.197104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.197243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.215600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:45.217475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:45.291331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 624/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:45.583637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632165d-3c6e-4f54-b2eb-dd2c3b542593 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.702038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632165d-3c6e-4f54-b2eb-dd2c3b542593 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.740694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.741325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.741604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.741876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.753917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:45.758849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:45.760070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:45.761040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 598/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:29:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:45.766646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.869466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947353a0-404e-46fb-8586-ef076bd7e249 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:45.892785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:45.905408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:45.906076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:45.906382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:45.906675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:45.907366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:45.907682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:45.909163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:45.918316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 625/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:29:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:45.924570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:45.973349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:46.040532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0c933-6909-4786-b279-1509a39af9ee None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:46.083049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:46.083684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:46.083986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:46.084265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:46.094579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:46.096126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:46.110756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:46.116691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:46.117741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:46.117741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:46.117741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:46.117741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 599/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:46.117978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:46.231250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b093db8-47a0-4ad0-b85f-8909f5b5cdb6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:46.267601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:46.268239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:46.268518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:46.268793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:46.269434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:46.269737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:46.271128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:46.284563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 626/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:46.291380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:46.401056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:46.411007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543952-9cb5-4689-8516-3fee6e1743cf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:46.455101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:46.455748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:46.456160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:46.456462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:46.465360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:46.467453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:46.478855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:46.484763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:46.486068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 600/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:46.486781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:46.538120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:46.601694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cd157-9f4c-4556-8931-4f3838860ce3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:46.641913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:46.642667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:46.642948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:46.643221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:46.643974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:46.644359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:46.646187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:46.666333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 627/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:46.672991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:46.782739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345892c-ddf5-41e8-bb7c-cc168e6c175b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:46.820380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:46.821011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:46.821287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:46.821558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:46.832657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:46.834113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:46.849088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:46.850224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:46.851187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:46.851187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:46.851187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:46.851187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 601/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:46.857268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:46.967221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4cc6e-95bd-4279-8a1e-41d018a12f50 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:46.969845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:47.006865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:47.007474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.007751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.008054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.008696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:47.009015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:47.010352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:47.023854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 628/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:47.030287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.041390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:47.137661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41e8de-9b87-472f-ad46-f4a7bb387b5e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:47.179291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:47.179925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.180237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.180513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.188919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:47.190458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:47.203303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:47.204682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:47.205695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 602/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:47.211281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.321467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bad81a-6b7e-4c07-bb73-486721c698b9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:47.359937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:47.360577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.360859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.361136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.361834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:47.362142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:47.363583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1cd3e-ce2f-43f5-b63e-dfa4c2d43d0a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:47.376556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 629/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:47.382957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:47.460931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:47.504262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fc8ce-bbee-492c-aa55-4548ce01765a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:47.545070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc9d14-447b-43d3-9d40-6a8b29aa15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:47.545798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc9d14-447b-43d3-9d40-6a8b29aa15ba None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:47.546150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc9d14-447b-43d3-9d40-6a8b29aa15ba None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:47.546452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc9d14-447b-43d3-9d40-6a8b29aa15ba None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/e605979196e14110a19a6aff30f2006d/roles/21da3a7fa5f2485d9b0bc4b91a29a59f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:47.563281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc9d14-447b-43d3-9d40-6a8b29aa15ba None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=e605979196e14110a19a6aff30f2006d, role_id=21da3a7fa5f2485d9b0bc4b91a29a59f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:47.564916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc9d14-447b-43d3-9d40-6a8b29aa15ba None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:47.579022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 603/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:47 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/e605979196e14110a19a6aff30f2006d/roles/21da3a7fa5f2485d9b0bc4b91a29a59f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:29:49.545228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ca953-467e-476c-bdf6-420b5741461e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.545793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ca953-467e-476c-bdf6-420b5741461e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.545929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ca953-467e-476c-bdf6-420b5741461e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.547284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 630/1233] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:49.552861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:49.553360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:49.553360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:49.574345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:49.576757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:49.651036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 604/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:49.961365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ca953-467e-476c-bdf6-420b5741461e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:49.993462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:50.081526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ca953-467e-476c-bdf6-420b5741461e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:50.118398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:50.119102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.119399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.119674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.130848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:50.135763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:50.142972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:50.143997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 631/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 16:29:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:50.144792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:50.196493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:50.264906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd6a9d-eb98-4737-be69-9fbeb49a86a6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:50.303729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:50.304505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.304795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.305083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.305848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:50.306177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:50.307535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:50.316470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 605/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 16:29:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:50.322276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:50.428929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:50.447003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fbf7a-0dea-437e-aacd-31418e5a77aa None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:50.488640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:50.489276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.489557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.489838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.500682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:50.502081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:50.517451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:50.518567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:50.519640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:50.519640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:50.519640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:50.519640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 632/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:50.525167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:50.642640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f099b7-7856-4163-8b8a-ace58bb75206 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:50.653239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:50.680354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:50.680986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.681291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.681607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.682283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:50.682588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:50.684085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:50.698124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 606/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:50.706577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:50.819679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec1c7f-004f-4156-81d7-07ad80843334 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:50.855883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:50.856934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:50.857190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:50.857476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:50.860180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:50.865806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:50.867542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:50.879538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:50.880721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:50.881903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 633/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:50.887369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:50.998917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9870cc4-8484-4bd3-8bb7-1786ca7e64ba None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.040770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.041387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.041668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.041942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.042682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:51.042990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.045343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.064434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 607/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:51.070617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:51.189039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dfc84-af34-4cfe-a33c-536b475adf89 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.230016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.230630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.231004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.231524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.302589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:51.310552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.312021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.426237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 634/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:51.433004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:51.468701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:51.550307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b86fd-997a-4d78-b18b-3372109ed501 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.588541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.589363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.589745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.590094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.603013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.604635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:51.624904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:51.625925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:51.627031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:51.627031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:51.627031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:51.627031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 608/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:51.632806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:51.742972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:51.749790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1ca9c-dab7-4f19-b462-63fa252f2378 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.795805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.796480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.796745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.797051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.797703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:51.798013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.799414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.812379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 635/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:51.818364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:51.903765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:51.927580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5ec2a-222a-4fa7-a306-ce06b4f312e7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:51.962525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:51.963164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:51.963444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:51.963719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:51.972355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:51.974158 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:51.989232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:51.990231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:51.991155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 609/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:51.998945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:52.108655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db3ded-3238-4f2a-aa58-7c42edfe2f04 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:52.145077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:52.145693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:52.146026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:52.146245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:52.147027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:52.147400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:52.149039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aba0c5-e966-420a-a10e-a4e769bc8f08 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:52.164496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 636/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:52.171959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:52.290930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6ef8b-e0bd-41ed-85f8-8b6f0c432a09 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:52.331183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:52.331817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:52.332327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:52.333235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:52.334570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] RBAC: Adding query filter params (role.id=21da3a7fa5f2485d9b0bc4b91a29a59f, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=e605979196e14110a19a6aff30f2006d) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:52.334985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:52.336710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a502f9a-4c77-468d-81f2-3cb5f82a282e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:52.345154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 610/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:52 2022] GET /identity/v3/role_assignments?user.id=e605979196e14110a19a6aff30f2006d&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=21da3a7fa5f2485d9b0bc4b91a29a59f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:54.593718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebcba3-3574-4aa4-b460-ac19d08805ad None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:54.594283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebcba3-3574-4aa4-b460-ac19d08805ad None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:54.594283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebcba3-3574-4aa4-b460-ac19d08805ad None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.595815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 637/1247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:54.601821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:54.602355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:54.602461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:54.610270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:54.623888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:54.625231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:54.702423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 611/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:55.002856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebcba3-3574-4aa4-b460-ac19d08805ad None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:55.023363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:55.122962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebcba3-3574-4aa4-b460-ac19d08805ad None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:55.163368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:55.164067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:55.164385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:55.164627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:55.176625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:55.181943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:55.182969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:55.183805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:55.188255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 638/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:55.189982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:55.303248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42e6ec-deea-472e-bf5d-86538d0dbb5d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:55.339027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:55.344270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:55.344926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:55.345217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:55.345835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:55.346551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:55.347118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:55.348736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:55.358238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 612/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:55.364003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:55.463817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:55.492509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbaf55-d783-4c80-a2fe-5476fc621e19 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:55.530534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:55.531163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:55.531444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:55.531719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:55.547009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:55.548619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:55.565560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:55.566754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:55.567664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:55.567664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:55.567664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:55.572264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 639/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:55.573138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:55.692250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea786a-b670-44aa-b1e9-aaade24498a6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:55.730047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:55.730684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:55.730993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:55.731337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:55.732145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:55.732457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:55.733936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:55.752697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 613/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:55.753993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:55.861524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c5c3d-ca90-4f27-a7ac-bdd0289a6488 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:55.890450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:55.900433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:55.901067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:55.901350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:55.901623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:55.910547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:55.912025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:55.923366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:55.924521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:55.925980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 640/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:55.930384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:55.991483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:56.041289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa8b45-4cfa-4293-ae02-c7303b477569 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:56.081575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:56.082279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.082583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.082879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.083668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:56.083990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:56.085413 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:56.104915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 614/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:56.110830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:56.224015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadda8f-bb45-4591-aed9-3a0541562bac None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:56.260830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22812990-458f-4d3c-8872-811406b8731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:56.261439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22812990-458f-4d3c-8872-811406b8731f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.261717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22812990-458f-4d3c-8872-811406b8731f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.261993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22812990-458f-4d3c-8872-811406b8731f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.328115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:56.342327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22812990-458f-4d3c-8872-811406b8731f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:56.343865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22812990-458f-4d3c-8872-811406b8731f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:56.455064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 641/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:56.461195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:56.570717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda392f2-7d9c-4ce9-8ce9-d3ff7a9a7c75 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:56.584665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:56.609173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:56.609953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.610388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.610719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.623524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:56.625387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:56.641884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:56.643116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:56.644278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:56.644278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:56.644278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:56.644278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 615/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:56.648758 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22812990-458f-4d3c-8872-811406b8731f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:56.751918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:56.766486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22812990-458f-4d3c-8872-811406b8731f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:56.805128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:56.805833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.806179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.806518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.807274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:56.807626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:56.810015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:56.824009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 642/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:56.830471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:56.945540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ad4f1-d8a2-47b2-9c45-c31790391692 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:56.984949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:56.985655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:56.985935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:56.986236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:56.994949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:56.996409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:57.011181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:57.012888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:29:57.013835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 616/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:57.019074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:57.133315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f86d-dd70-44e9-80c5-bf2257d6ac72 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:57.175443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:57.176090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:57.176384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:57.176647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:57.177340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:57.177650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:57.179066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79575ad3-942f-4992-9190-cbc8f8ca0520 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.197534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 643/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:57.198771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:57.227962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:57.312556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d535e4e-cded-42e5-b290-bd51da85857b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:57.350949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:57.351690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:57.351902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:57.352240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:57.353009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=e605979196e14110a19a6aff30f2006d) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:57.353388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:57.355001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea21eea-3899-4aaf-b4ad-998358563641 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:57.360785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 617/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:57 2022] GET /identity/v3/role_assignments?user.id=e605979196e14110a19a6aff30f2006d&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=580c9e29995a41c49a5a943d7b48532c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:59.459321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd761c3e-1c22-4904-a2b7-4593fe46b288 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:59.459851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd761c3e-1c22-4904-a2b7-4593fe46b288 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:59.459851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd761c3e-1c22-4904-a2b7-4593fe46b288 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:59.461255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 644/1261] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:59.466802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:59.467291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:59.467381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:59.486028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:59.487212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:59.552316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 618/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:59.819802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd761c3e-1c22-4904-a2b7-4593fe46b288 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:59.830844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:59.919650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd761c3e-1c22-4904-a2b7-4593fe46b288 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:59.953252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:59.953867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:59.954148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:59.954425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:59.964207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:59.968865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:59.970018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:29:59.971125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 645/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:59.975861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:59.994328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.076105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173dcc7-ad64-4c38-9eb2-f4fac12047af None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.111254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.111856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.112165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.112441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.113102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:00.113406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.114770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.123379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 619/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:00.129158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.198074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.233683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5737db-9534-4923-8908-eed4f0e649ab None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.265889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.266500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.266779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.267054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.277200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.279554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:30:00.294223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:00.295501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:30:00.297488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:30:00.297488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:00.297488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:30:00.297488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 646/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:00.303374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.381622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.401094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35e433-528e-42bd-87ba-7f4992769150 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.441802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.442468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.442851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.443171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.443799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:00.444192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.445791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.460256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 620/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:00.466842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.570078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780d854-85dd-4ae8-a0be-c1791ee940e7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.589727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.603720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.604378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.604662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.604938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] PATH_INFO: `/v3/users/placement` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.612953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.614378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:30:00.625847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:00.631844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:30:00.633120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 647/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:30:00.633901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.729346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982696b-8988-4d3f-9318-9450835bf274 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.762842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.763434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.763711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.764000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.764718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:00.765018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.766412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.774675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:00.783270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 621/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:00.789245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:00.886625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dafa7-f2da-4d47-ad4d-eb3eb1c748da None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:00.919846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:00.920474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:00.920749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:00.921017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:00.931075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:00.932533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:30:00.945945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:00.947077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:30:00.948367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:30:00.948367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:00.948367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:30:00.948367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 648/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:00.953062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:01.051731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f5129-1fff-42ff-909c-d6477dfcb455 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:01.085745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:01.086343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:01.086621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:01.086893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:01.087529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:01.087830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:01.089224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:01.100689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 622/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:01.106909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:01.138581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:01.207235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3a2a0-4ccd-4bb3-97f7-5fb1d7dc0ccb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:01.240840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:01.241458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:01.241737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:01.242010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:01.249052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:01.250500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:30:01.262885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:01.264017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:30:01.265061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 649/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:30:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:01.269815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:01.328320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:01.373209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f95bc3-3170-4c5c-b3c0-de01801574c3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:01.408574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:01.409217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:01.409535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:01.409832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:01.410613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:01.410963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:01.412835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af48b2-922f-437f-a37a-950967a409e2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:01.426193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 623/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:01.432919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:01.524777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:01.534167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09442df-1262-44c2-b3c5-0511fc2e46fa None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:01.566950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5ef41-6695-4a78-a7bf-3f8211480687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:01.567603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5ef41-6695-4a78-a7bf-3f8211480687 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:01.567884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5ef41-6695-4a78-a7bf-3f8211480687 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:01.568189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5ef41-6695-4a78-a7bf-3f8211480687 None admin] PATH_INFO: `/v3/projects/77eff9a0065049a29630adf71df889d3/users/e605979196e14110a19a6aff30f2006d/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:01.570074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5ef41-6695-4a78-a7bf-3f8211480687 None admin] RBAC: Authorizing `identity:create_grant(project_id=77eff9a0065049a29630adf71df889d3, user_id=e605979196e14110a19a6aff30f2006d, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:01.571495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5ef41-6695-4a78-a7bf-3f8211480687 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:01.584292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 650/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 16:30:01 2022] PUT /identity/v3/projects/77eff9a0065049a29630adf71df889d3/users/e605979196e14110a19a6aff30f2006d/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:30:03.453153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c69a1-4d67-49c9-9ac9-3fbdd9d86043 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:03.453696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c69a1-4d67-49c9-9ac9-3fbdd9d86043 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:03.453696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c69a1-4d67-49c9-9ac9-3fbdd9d86043 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:03.455059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 624/1274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:03.460261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:03.460684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:03.460801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:03.478630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:03.479808 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:03.546871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 651/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:03.814399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c69a1-4d67-49c9-9ac9-3fbdd9d86043 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:03.914787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c69a1-4d67-49c9-9ac9-3fbdd9d86043 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:03.931371 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:03.948535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:03.949160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:03.950116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:03.950395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:03.960183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:30:03.964842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:03.965940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:03.967032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 625/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 16:30:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:03.972151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:04.053404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:04.072901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a922ce0-093a-4d2b-99aa-60f68bbc31be None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:04.107145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:04.107752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:04.108065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:04.108345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:04.109007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:04.109311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:04.110679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:04.118472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 652/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 16:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:04.124843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:04.223732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed744e98-4c78-4a2e-bb39-12c43a05ad49 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:04.258244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:04.258864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:04.259141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:04.259414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:04.269555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:04.270990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:30:04.285327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:04.286438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:30:04.287812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:30:04.287812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:04.287812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:04.287812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 626/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:04.292550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:04.396314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b14e46-b5a5-4ff3-82a6-72d00adb29b0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:04.444523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:04.445267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:04.445662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:04.446048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:04.446901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:04.447287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:04.448830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:04.462826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 653/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:04.469696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:04.502259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:04.569135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6111e-057c-4647-adac-fde947cc6ec2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:04.600916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:04.601521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:04.601797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:04.602066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:04.609747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:04.611165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:30:04.622349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:04.623461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:04.624473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 627/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:30:04.629281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:04.632978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:04.728223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4411ee-187e-424f-8a14-4c2b57c6130f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:04.761188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:04.761838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:04.762114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:04.762385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:04.763110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:04.763417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:04.764856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:04.782519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 654/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 16:30:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:04.789269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:04.896064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:04.907927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4a221-e65a-4840-8250-ce1d354d3893 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:04.946310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:04.946921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:04.947202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:04.947477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.028273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.029664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.127288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:05.142390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 628/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 16:30:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:30:05.148501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.211232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:05.249445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f332a0-f5af-4393-96ac-50f1693cdcc0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:05.282820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:05.283373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.283653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.283926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.293921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.295349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:30:05.310237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:05.311395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:30:05.312531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:30:05.312531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:30:05.312531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:30:05.312531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 655/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:30:05.318646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.416361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4b7c-4fe2-4f85-871f-74e8591312bc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:05.450382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:05.450974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.451248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.451517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.452173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:05.452472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.453817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.464548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 629/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 16:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:05.470285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.569572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9fc7e-f68e-42ba-9ae4-6a45ae09c3dc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:05.583546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:05.603666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:05.604314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.604592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.604861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.611677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.613142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:30:05.624898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:05.625783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Aug 24 16:30:05.626875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 656/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 16:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:05.631751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.681193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:05.731790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e3cae-5ebc-41b6-8859-46e9d54283e5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:05.768441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:05.769047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.769334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.769608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.770298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:05.770603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.772039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef715c-e4c0-41ea-a2e0-1e03403313fd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.783794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 630/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 16:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:05.790177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:05.887801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf186258-515a-411b-a0ab-96227267a310 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:05.920839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:05.921430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:05.921713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:05.921989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:05.922711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] RBAC: Adding query filter params (role.id=580c9e29995a41c49a5a943d7b48532c, scope.project.id=77eff9a0065049a29630adf71df889d3, user.id=e605979196e14110a19a6aff30f2006d) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:05.923024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:05.924481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e361b-0e1b-40ee-9ec6-e92c5d85430b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:05.930111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 657/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 16:30:05 2022] GET /identity/v3/role_assignments?user.id=e605979196e14110a19a6aff30f2006d&scope.project.id=77eff9a0065049a29630adf71df889d3&role.id=580c9e29995a41c49a5a943d7b48532c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:07.841531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b06fc-25d8-4b5a-af2e-f41490792d53 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:07.842100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b06fc-25d8-4b5a-af2e-f41490792d53 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:07.842100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b06fc-25d8-4b5a-af2e-f41490792d53 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:07.843433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 631/1288] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:07.848507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:07.848906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:07.849055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:07.866581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:07.867745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:07.897105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:07.933266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 658/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:08.202266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b06fc-25d8-4b5a-af2e-f41490792d53 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:08.300615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b06fc-25d8-4b5a-af2e-f41490792d53 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:08.311017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:08.337803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:08.338434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.338850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.339161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.340864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:08.342244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:08.350764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:08.351992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:08.352881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 632/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:30:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:08.358475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:08.463619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc918-6387-4112-a35b-11b133f52f9b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:08.499876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:08.500514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.500798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.501074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.501700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:08.502006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:08.503365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b54d0-06a4-455a-8ae5-37c2c7f73fa3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:08.508072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 659/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 16:30:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:08.514488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:08.613642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf6175-8ca2-474b-bcb7-b9506497e622 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:08.647068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:08.647660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:08.647936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:08.648233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:08.648850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:08.649149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:08.650498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d15bd-4e79-4e6c-8c39-58d7e003ba50 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:08.654978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 633/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 16:30:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:10.527544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561bee-c2f6-44f4-9f0a-669d25c28f45 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:10.528091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561bee-c2f6-44f4-9f0a-669d25c28f45 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:10.528270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7561bee-c2f6-44f4-9f0a-669d25c28f45 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:10.529614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 660/1293] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:10.535007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa39767-fbb8-43b2-962a-60c5c2b18373 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:10.535390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa39767-fbb8-43b2-962a-60c5c2b18373 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:10.535563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa39767-fbb8-43b2-962a-60c5c2b18373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:10.554209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eaa39767-fbb8-43b2-962a-60c5c2b18373 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:10.555458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eaa39767-fbb8-43b2-962a-60c5c2b18373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:10.584573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:10.628729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 634/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:10.895572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7561bee-c2f6-44f4-9f0a-669d25c28f45 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:10.927109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:10.996194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7561bee-c2f6-44f4-9f0a-669d25c28f45 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:11.030579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea53e2-216f-4b47-b464-5261065eaa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:11.031179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea53e2-216f-4b47-b464-5261065eaa76 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:11.031453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea53e2-216f-4b47-b464-5261065eaa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:11.031720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea53e2-216f-4b47-b464-5261065eaa76 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:11.032398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea53e2-216f-4b47-b464-5261065eaa76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:11.033735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea53e2-216f-4b47-b464-5261065eaa76 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:11.048214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 661/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 16:30:10 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:30:12.954745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b3f19-2af3-46dd-a958-53c5b78bf048 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.955302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b3f19-2af3-46dd-a958-53c5b78bf048 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.955302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b3f19-2af3-46dd-a958-53c5b78bf048 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.956668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 635/1296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:12.961824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1beb140-88d4-408e-a079-53989a67777d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:12.962218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1beb140-88d4-408e-a079-53989a67777d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:12.962404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1beb140-88d4-408e-a079-53989a67777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:12.980667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f1beb140-88d4-408e-a079-53989a67777d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:12.981909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f1beb140-88d4-408e-a079-53989a67777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:13.054394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 662/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:13.322094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b3f19-2af3-46dd-a958-53c5b78bf048 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:13.424226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b3f19-2af3-46dd-a958-53c5b78bf048 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:13.457063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:13.457681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:13.457960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:13.458230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:13.458810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:13.460191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:13.469143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:13.470279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:13.471327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 636/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:30:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:13.476393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1beb140-88d4-408e-a079-53989a67777d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:13.494503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:13.574611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1beb140-88d4-408e-a079-53989a67777d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:13.607410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:13.608042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:13.608324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:13.608597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:13.609218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:13.609521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:13.610878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:13.615819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 663/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 16:30:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:13.622520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:13.653575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:13.721830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f26758-7cf0-4a25-bcb1-ab312e690bd9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:13.754051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:13.754644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:13.754921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:13.755197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:13.755886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] RBAC: Adding query filter params (interface=public, service_id=3b3ed737412f402b869ed9a7ac5d102a, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:13.756218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:13.757575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e447b-767f-4829-afc7-0ac4373de26e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:13.763039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 637/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 16:30:13 2022] GET /identity/v3/endpoints?service_id=3b3ed737412f402b869ed9a7ac5d102a&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:13.768733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:13.847576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:13.864995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8148e-5501-4a7d-9169-1094a42cdb90 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:13.897865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd6d0b-6950-4626-bb3e-699b27a5ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:13.898456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6d0b-6950-4626-bb3e-699b27a5ed27 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:13.898731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6d0b-6950-4626-bb3e-699b27a5ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:13.899002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd6d0b-6950-4626-bb3e-699b27a5ed27 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:13.899592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd6d0b-6950-4626-bb3e-699b27a5ed27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:13.900970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd6d0b-6950-4626-bb3e-699b27a5ed27 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:13.907763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 664/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 16:30:13 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:30:15.822611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e2e9d-afc6-4db4-8a94-83c03edb5ba5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.823139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e2e9d-afc6-4db4-8a94-83c03edb5ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.823139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e2e9d-afc6-4db4-8a94-83c03edb5ba5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.825112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 638/1302] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:30:15.830563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:15.830977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:15.831114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:15.850099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:30:15.851262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:30:15.916679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 665/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:30:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:30:16.193550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e2e9d-afc6-4db4-8a94-83c03edb5ba5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:16.293032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:16.300386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e2e9d-afc6-4db4-8a94-83c03edb5ba5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:16.334564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:16.335281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:16.335627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:16.335906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:16.336519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:16.337887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:30:16.347565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:30:16.348606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Aug 24 16:30:16.349470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 639/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 16:30:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:30:16.354881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:16.438532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:16.463885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e05a0c-aed9-4f2e-ad37-aa9e98c283b0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:16.498682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:16.499291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:16.499569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:16.499852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:16.500516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:30:16.500819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:16.502179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58c818-fcc2-4b36-9ee7-5ac5468ac867 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:16.507469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 666/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 16:30:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:30:16.515506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:30:16.617418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afed4f-d76e-4e56-a936-140b5821a5cc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:30:16.652225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bea59-9370-430a-beee-bd0aaae4a7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:30:16.652816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bea59-9370-430a-beee-bd0aaae4a7ca None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:30:16.653088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bea59-9370-430a-beee-bd0aaae4a7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:30:16.653353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bea59-9370-430a-beee-bd0aaae4a7ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:30:16.654001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bea59-9370-430a-beee-bd0aaae4a7ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:30:16.655346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bea59-9370-430a-beee-bd0aaae4a7ca None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:30:16.674434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 640/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:30:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:30:26.504394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=30 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:30:26.670163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:21.263494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69267f-5809-4e11-a0dc-2ad8dceb7af6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:21.263999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69267f-5809-4e11-a0dc-2ad8dceb7af6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:21.264244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69267f-5809-4e11-a0dc-2ad8dceb7af6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:21.268365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 667/1307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:21.270990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed001d-8da7-4c60-ba71-2cf18dba0caf None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:21.271384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed001d-8da7-4c60-ba71-2cf18dba0caf None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:21.271483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed001d-8da7-4c60-ba71-2cf18dba0caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:21.362891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a8ed001d-8da7-4c60-ba71-2cf18dba0caf None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:21.364694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a8ed001d-8da7-4c60-ba71-2cf18dba0caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:21.536654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 641/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:21.839068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e42e0-f174-4a6a-ae7c-ce79f8767847 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:21.839415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e42e0-f174-4a6a-ae7c-ce79f8767847 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:21.839625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e42e0-f174-4a6a-ae7c-ce79f8767847 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:21.841248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 668/1309] 10.222.0.52 () {54 vars in 965 bytes} [Wed Aug 24 16:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:21.847377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d3ded-86de-4080-b738-ea327a9cbc45 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:21.848063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d3ded-86de-4080-b738-ea327a9cbc45 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:21.848659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d3ded-86de-4080-b738-ea327a9cbc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:21.934289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a86d3ded-86de-4080-b738-ea327a9cbc45 None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:21.936106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a86d3ded-86de-4080-b738-ea327a9cbc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:21.961377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:22.062747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 642/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Wed Aug 24 16:31:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:22.071835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e42e0-f174-4a6a-ae7c-ce79f8767847 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:22.073619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2d2e42e0-f174-4a6a-ae7c-ce79f8767847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:22.207983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e42e0-f174-4a6a-ae7c-ce79f8767847 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:22.299731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:22.300393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:22.300643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:22.301122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:22.302890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:22.362297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:22.425254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:22.426906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89534a9e-741b-4183-af9b-cf960f66b67d service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:22.468412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 669/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:24.741639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adc70b-fe0b-46ea-b31c-42c4b1802286 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:24.742237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adc70b-fe0b-46ea-b31c-42c4b1802286 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:24.742237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adc70b-fe0b-46ea-b31c-42c4b1802286 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:24.743567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 643/1312] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:24.750531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfb15-4293-47f6-96e6-179e7a15a6f1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:24.751065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfb15-4293-47f6-96e6-179e7a15a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:24.751186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfb15-4293-47f6-96e6-179e7a15a6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:24.833494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c20cfb15-4293-47f6-96e6-179e7a15a6f1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:24.834684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c20cfb15-4293-47f6-96e6-179e7a15a6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:24.935099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 670/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:25.218140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adc70b-fe0b-46ea-b31c-42c4b1802286 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:25.219704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-61adc70b-fe0b-46ea-b31c-42c4b1802286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:25.323329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adc70b-fe0b-46ea-b31c-42c4b1802286 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:25.394827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:25.397591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:25.398380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:25.398857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:25.399352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:25.401259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:25.509339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:25.510954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00105a8e-3a9e-405a-8c5d-bc2746b93d49 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:25.528914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 644/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:27.773980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b75856-3a66-4ed2-a673-61154c09601e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:27.774548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b75856-3a66-4ed2-a673-61154c09601e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:27.774548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b75856-3a66-4ed2-a673-61154c09601e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:27.777916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 671/1315] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:27.783507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ebf7-8312-4f4f-bc8e-ffd28157115e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:27.784011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ebf7-8312-4f4f-bc8e-ffd28157115e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:27.784107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ebf7-8312-4f4f-bc8e-ffd28157115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:27.852425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8a32ebf7-8312-4f4f-bc8e-ffd28157115e None None] MFA Rules not processed for user `7db0d9ecb6fa4688a758312e6c7cd99a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:27.854275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8a32ebf7-8312-4f4f-bc8e-ffd28157115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:27.920714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:27.992796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 645/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:28.304461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60b546-4092-4ccd-b444-8882ef081e3c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:28.304670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60b546-4092-4ccd-b444-8882ef081e3c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:28.304929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60b546-4092-4ccd-b444-8882ef081e3c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:28.306277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 672/1317] 10.222.0.52 () {54 vars in 965 bytes} [Wed Aug 24 16:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:28.312872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc9acb-e79f-4b20-b14f-ee4067834c86 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:28.313157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc9acb-e79f-4b20-b14f-ee4067834c86 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:28.314021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc9acb-e79f-4b20-b14f-ee4067834c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:28.404152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-37fc9acb-e79f-4b20-b14f-ee4067834c86 None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:28.405763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-37fc9acb-e79f-4b20-b14f-ee4067834c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:28.482976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 646/1318] 10.222.0.52 () {58 vars in 1062 bytes} [Wed Aug 24 16:31:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:28.492155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60b546-4092-4ccd-b444-8882ef081e3c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:28.493702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9c60b546-4092-4ccd-b444-8882ef081e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:28.504512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:28.598682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60b546-4092-4ccd-b444-8882ef081e3c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:28.670380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:28.670981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:28.671258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:28.671531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:28.673145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:28.733951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:28.780587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:28.782018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41164f-703e-4014-ab67-40e7c8ef116a service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:28.801203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 673/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:31.604074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522c34-7c22-4b42-933a-4e449a7d4c74 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:31.604619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522c34-7c22-4b42-933a-4e449a7d4c74 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:31.604619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0522c34-7c22-4b42-933a-4e449a7d4c74 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:31.605967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 647/1320] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:31.611601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b48d0-8eb1-4f8b-909d-87b1af395e90 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:31.612051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b48d0-8eb1-4f8b-909d-87b1af395e90 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:31.612248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b48d0-8eb1-4f8b-909d-87b1af395e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:31.677434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a45b48d0-8eb1-4f8b-909d-87b1af395e90 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:31.678679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a45b48d0-8eb1-4f8b-909d-87b1af395e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:31.776308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 674/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:32.058787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522c34-7c22-4b42-933a-4e449a7d4c74 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:32.060389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c0522c34-7c22-4b42-933a-4e449a7d4c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:32.112568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:32.166445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0522c34-7c22-4b42-933a-4e449a7d4c74 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:32.237301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:32.237903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:32.238177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:32.238444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:32.240059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:32.333037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:32.354053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:32.355504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327d403-7ccf-4ae6-b03f-386a36d5a4c3 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:32.373385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 648/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:31:35.090930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df33c7-f42b-402c-85f4-25000b802727 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:35.091478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df33c7-f42b-402c-85f4-25000b802727 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:35.091478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df33c7-f42b-402c-85f4-25000b802727 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:35.095337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 675/1323] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:35.098157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe9db0-23f7-48a5-883c-4564a5120fca None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:35.098583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe9db0-23f7-48a5-883c-4564a5120fca None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:35.098692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe9db0-23f7-48a5-883c-4564a5120fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:35.165010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-fdbe9db0-23f7-48a5-883c-4564a5120fca None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:35.166721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fdbe9db0-23f7-48a5-883c-4564a5120fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:35.266928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 649/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:35.558962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df33c7-f42b-402c-85f4-25000b802727 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:35.560541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-80df33c7-f42b-402c-85f4-25000b802727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:35.579903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:35.664600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df33c7-f42b-402c-85f4-25000b802727 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:35.737807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:35.738326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:35.738599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:35.738865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:35.740476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:35.811745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:35.850762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:35.852263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745577f-0173-47bb-9527-c1156f2525c7 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:35.870082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 676/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:37.856894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbadc17-d7c1-4720-a0f6-1d14dc2f36c5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:37.857442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbadc17-d7c1-4720-a0f6-1d14dc2f36c5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:37.857442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbadc17-d7c1-4720-a0f6-1d14dc2f36c5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:37.858805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 650/1326] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:31:37.864169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7440-ce76-4777-953f-d77c3b542c6e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:37.864588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7440-ce76-4777-953f-d77c3b542c6e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:37.864727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7440-ce76-4777-953f-d77c3b542c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:37.932552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b1ac7440-ce76-4777-953f-d77c3b542c6e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:37.933766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b1ac7440-ce76-4777-953f-d77c3b542c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:38.033395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 677/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:31:38.314386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbadc17-d7c1-4720-a0f6-1d14dc2f36c5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:38.315977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0fbadc17-d7c1-4720-a0f6-1d14dc2f36c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:38.379059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:38.433095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbadc17-d7c1-4720-a0f6-1d14dc2f36c5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:38.507798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:38.508437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:38.508718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:38.508994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:38.510600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:38.619913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:38.622261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:38.624124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95adf8-7e80-498c-aae3-8d056961f82b service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:38.642660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 651/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:40.638125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588916ab-20f8-4c75-9562-a1139f76685d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:40.638711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588916ab-20f8-4c75-9562-a1139f76685d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:40.638711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588916ab-20f8-4c75-9562-a1139f76685d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:40.640063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 678/1329] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:40.645923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8028c16-2dba-4755-89a0-abaa0b76ee0b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:40.646479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8028c16-2dba-4755-89a0-abaa0b76ee0b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:40.646479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8028c16-2dba-4755-89a0-abaa0b76ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:40.713978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e8028c16-2dba-4755-89a0-abaa0b76ee0b None None] MFA Rules not processed for user `7db0d9ecb6fa4688a758312e6c7cd99a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:40.715176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e8028c16-2dba-4755-89a0-abaa0b76ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:40.817995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 652/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:41.111312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588916ab-20f8-4c75-9562-a1139f76685d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:41.112982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-588916ab-20f8-4c75-9562-a1139f76685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:41.153523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:41.239568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588916ab-20f8-4c75-9562-a1139f76685d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:41.316625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:41.317183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:41.317524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:41.317709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:41.319340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:41.411489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:41.451976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:41.453505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963fd44-29bb-495e-9591-0aa9dad3f1da service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:41.471803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 679/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:43.628448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa549cf-5271-4681-9dd9-ba2cb5d495f8 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:43.628973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa549cf-5271-4681-9dd9-ba2cb5d495f8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:43.628973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa549cf-5271-4681-9dd9-ba2cb5d495f8 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:43.630331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 653/1332] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:43.635634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09235-bd54-43ea-8f52-ff3d9ef06f57 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:43.636131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09235-bd54-43ea-8f52-ff3d9ef06f57 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:43.636247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09235-bd54-43ea-8f52-ff3d9ef06f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:43.701908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d9b09235-bd54-43ea-8f52-ff3d9ef06f57 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:43.703111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d9b09235-bd54-43ea-8f52-ff3d9ef06f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:43.801514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 680/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:44.093454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa549cf-5271-4681-9dd9-ba2cb5d495f8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:44.095042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7fa549cf-5271-4681-9dd9-ba2cb5d495f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:44.164712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:44.198394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa549cf-5271-4681-9dd9-ba2cb5d495f8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:44.270823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:44.271430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:44.271709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:44.272005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:44.273613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:44.382330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:44.384370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a043a8-5b6b-431d-9125-3850d94d0fcd service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:44.394183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:44.402704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 654/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:46.372798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a0bd-409a-480e-8ef3-08ce289e60a3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:46.373339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a0bd-409a-480e-8ef3-08ce289e60a3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:46.373339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942a0bd-409a-480e-8ef3-08ce289e60a3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:46.374732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 681/1335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:46.379850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0e97a-2ffe-4195-9170-27cae7a3346d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:46.380296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0e97a-2ffe-4195-9170-27cae7a3346d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:46.380407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0e97a-2ffe-4195-9170-27cae7a3346d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:46.446327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-51c0e97a-2ffe-4195-9170-27cae7a3346d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:46.447601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-51c0e97a-2ffe-4195-9170-27cae7a3346d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:46.543241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 655/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:46.827231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942a0bd-409a-480e-8ef3-08ce289e60a3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:46.828832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7942a0bd-409a-480e-8ef3-08ce289e60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:46.868500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:46.934988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942a0bd-409a-480e-8ef3-08ce289e60a3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:47.008362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:47.009252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:47.009908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:47.010221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:47.011828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:47.096296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:47.122246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:47.123669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919026e-6e19-4cae-bfb2-75880f94afe0 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:47.141141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 682/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:49.380626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df19139-6e70-46a6-9026-0d936aa27871 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:49.381180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df19139-6e70-46a6-9026-0d936aa27871 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:49.381180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df19139-6e70-46a6-9026-0d936aa27871 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:49.382511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 656/1338] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:49.387600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf577-ab4f-4242-b54b-65bc94f1d2b5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:49.388070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf577-ab4f-4242-b54b-65bc94f1d2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:49.388177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bf577-ab4f-4242-b54b-65bc94f1d2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:49.454842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7d3bf577-ab4f-4242-b54b-65bc94f1d2b5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:49.456174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7d3bf577-ab4f-4242-b54b-65bc94f1d2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:49.551444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 683/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:49.834590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df19139-6e70-46a6-9026-0d936aa27871 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:49.836169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5df19139-6e70-46a6-9026-0d936aa27871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:49.911972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:49.940771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df19139-6e70-46a6-9026-0d936aa27871 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:50.013515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:50.014140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:50.014424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:50.014696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:50.016336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:50.124495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:50.125938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec268c7-ae8a-49e6-a965-139f444196fa service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:50.144138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 657/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:54.161744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738916f-a9cc-4c19-8f1a-27c8257f2f4b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:54.162271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738916f-a9cc-4c19-8f1a-27c8257f2f4b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:54.162389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738916f-a9cc-4c19-8f1a-27c8257f2f4b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:54.163805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 684/1341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:54.169722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a94ed-a89a-4ba3-bcab-4c29ca33d489 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:54.170205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a94ed-a89a-4ba3-bcab-4c29ca33d489 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:54.170308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a94ed-a89a-4ba3-bcab-4c29ca33d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:54.175754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:54.241720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b30a94ed-a89a-4ba3-bcab-4c29ca33d489 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:54.243216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b30a94ed-a89a-4ba3-bcab-4c29ca33d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:54.341317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 658/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:54.628181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738916f-a9cc-4c19-8f1a-27c8257f2f4b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:54.629803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f738916f-a9cc-4c19-8f1a-27c8257f2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:54.678419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:54.735203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738916f-a9cc-4c19-8f1a-27c8257f2f4b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:54.809485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:54.810093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:54.810376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:54.810648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:54.812264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:54.912519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:54.930672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:54.932204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9a36-3fb4-400f-800a-a4ffbb056d86 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:54.950762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 685/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:31:57.494472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0633f9-21b0-4af0-a95f-1532c909b2e4 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.495024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0633f9-21b0-4af0-a95f-1532c909b2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.495024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0633f9-21b0-4af0-a95f-1532c909b2e4 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.496407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 659/1344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:31:57.501866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d900eb-ec46-4175-ba27-af5ace3e0e87 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:57.502300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d900eb-ec46-4175-ba27-af5ace3e0e87 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:57.502407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d900eb-ec46-4175-ba27-af5ace3e0e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:57.572047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d2d900eb-ec46-4175-ba27-af5ace3e0e87 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:31:57.573273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d2d900eb-ec46-4175-ba27-af5ace3e0e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:57.669796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 686/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:31:57.958360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0633f9-21b0-4af0-a95f-1532c909b2e4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:31:57.959965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6b0633f9-21b0-4af0-a95f-1532c909b2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:58.053775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:31:58.072416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0633f9-21b0-4af0-a95f-1532c909b2e4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:31:58.147521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:31:58.148156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:31:58.148439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:31:58.149254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:31:58.150855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:31:58.269419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:31:58.270772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab8f3b-5310-42e5-94b4-be1d5861956e service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:31:58.290349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 660/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:31:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:02.466187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238adbd8-b3c3-4941-9a55-b0b2bfbaff07 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:02.466719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238adbd8-b3c3-4941-9a55-b0b2bfbaff07 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:02.466719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238adbd8-b3c3-4941-9a55-b0b2bfbaff07 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:02.468148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 687/1347] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:02.473761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c8b-e796-4836-a350-fd56b54a3dd1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:02.474260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c8b-e796-4836-a350-fd56b54a3dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:02.474355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c8b-e796-4836-a350-fd56b54a3dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:02.491082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:02.547015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-79641c8b-e796-4836-a350-fd56b54a3dd1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:02.548187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-79641c8b-e796-4836-a350-fd56b54a3dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:02.653328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 661/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:02.944367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238adbd8-b3c3-4941-9a55-b0b2bfbaff07 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:02.945933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-238adbd8-b3c3-4941-9a55-b0b2bfbaff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:03.005122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:03.052946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238adbd8-b3c3-4941-9a55-b0b2bfbaff07 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:03.135794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:03.136744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:03.136744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:03.137006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:03.138652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:03.250034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:03.253992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:03.255672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63009-b1d2-4c4f-b66d-d1c4d28b57fd service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:03.276398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 688/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:08.203886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec982c2-60aa-4b24-810b-541152873966 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:08.204628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec982c2-60aa-4b24-810b-541152873966 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:08.208241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec982c2-60aa-4b24-810b-541152873966 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:08.209126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 662/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:32:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:08.214125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03f66-dc2e-4c03-91b7-6e748012009e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:08.215051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03f66-dc2e-4c03-91b7-6e748012009e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:08.215487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03f66-dc2e-4c03-91b7-6e748012009e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:08.327560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-16c03f66-dc2e-4c03-91b7-6e748012009e None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:08.328829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-16c03f66-dc2e-4c03-91b7-6e748012009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:08.471806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 689/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:12.651358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=59 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:12.808105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c09f-0d4c-435c-a9d5-963cb38616c3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:12.808445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c09f-0d4c-435c-a9d5-963cb38616c3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:12.808740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8c09f-0d4c-435c-a9d5-963cb38616c3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:12.810083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 663/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:12.815261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3936c-c24c-447a-a44c-85e2e91f5a09 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:12.815710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3936c-c24c-447a-a44c-85e2e91f5a09 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:12.815853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3936c-c24c-447a-a44c-85e2e91f5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:12.875640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:12.902804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fdb3936c-c24c-447a-a44c-85e2e91f5a09 None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:12.904125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fdb3936c-c24c-447a-a44c-85e2e91f5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:13.024702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 690/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:14.411602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07df4b1-783a-49f0-8bfe-dd2e93e22734 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.412266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07df4b1-783a-49f0-8bfe-dd2e93e22734 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.412266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07df4b1-783a-49f0-8bfe-dd2e93e22734 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.414272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 664/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:14.420166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305615d-54c7-4229-ab91-48506175a8db None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:14.420623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305615d-54c7-4229-ab91-48506175a8db None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:14.420844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305615d-54c7-4229-ab91-48506175a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:14.497178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c305615d-54c7-4229-ab91-48506175a8db None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:14.498396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c305615d-54c7-4229-ab91-48506175a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:14.542255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:14.610305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 691/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:32:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:18.003083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275abca-edf4-441e-b03f-6b78953d8f14 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.003666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275abca-edf4-441e-b03f-6b78953d8f14 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.003666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275abca-edf4-441e-b03f-6b78953d8f14 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.005026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 665/1356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:18.011056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4ec0f-7721-4e3d-87d3-fb2714163943 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.011898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4ec0f-7721-4e3d-87d3-fb2714163943 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.012410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4ec0f-7721-4e3d-87d3-fb2714163943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.096461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a4d4ec0f-7721-4e3d-87d3-fb2714163943 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:18.097680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a4d4ec0f-7721-4e3d-87d3-fb2714163943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:18.227591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 692/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:18.554905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ca41-f5ce-4706-be7e-758c77e954fe None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.556082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ca41-f5ce-4706-be7e-758c77e954fe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.556561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02ca41-f5ce-4706-be7e-758c77e954fe None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.558602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 666/1358] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 16:32:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:18.568361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772924-7bbd-4b03-8c8b-d0fa0d71a17b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:18.568762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772924-7bbd-4b03-8c8b-d0fa0d71a17b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:18.569279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772924-7bbd-4b03-8c8b-d0fa0d71a17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:18.588875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:18.674459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5f772924-7bbd-4b03-8c8b-d0fa0d71a17b None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:18.675652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5f772924-7bbd-4b03-8c8b-d0fa0d71a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:18.800138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 693/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 16:32:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:18.809254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ca41-f5ce-4706-be7e-758c77e954fe None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:18.810987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ca02ca41-f5ce-4706-be7e-758c77e954fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:18.976817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02ca41-f5ce-4706-be7e-758c77e954fe None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:19.067848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:19.113680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:19.114293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:19.114576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:19.114853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:19.117321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:19.213301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:19.214912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c6031-5fb1-4c74-94ec-429003ee8a75 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:19.234377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 667/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:32:21.399625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711a126-0e71-4e67-838a-b5f93436f685 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:21.400218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711a126-0e71-4e67-838a-b5f93436f685 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:21.400218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711a126-0e71-4e67-838a-b5f93436f685 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:21.401620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 694/1361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:32:21.407179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f013-280c-452a-8805-3249a064e281 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:21.407906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f013-280c-452a-8805-3249a064e281 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:21.408013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f013-280c-452a-8805-3249a064e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:21.478485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4eb5f013-280c-452a-8805-3249a064e281 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:21.479768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4eb5f013-280c-452a-8805-3249a064e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:21.519075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:21.590531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 668/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:32:21.886263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711a126-0e71-4e67-838a-b5f93436f685 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:21.887775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6711a126-0e71-4e67-838a-b5f93436f685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:21.962297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:22.051655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711a126-0e71-4e67-838a-b5f93436f685 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:22.173795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:22.174256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:22.174709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:22.175164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:22.176859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:22.204960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:22.276497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:22.278094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2b97e-7036-4dfb-b8fb-f753399b44c0 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:22.298371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 695/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:24.577723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e977d-ddd8-49f1-a419-348fc3a9de8b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:24.578294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e977d-ddd8-49f1-a419-348fc3a9de8b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:24.578294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e977d-ddd8-49f1-a419-348fc3a9de8b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:24.581821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 669/1364] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:24.586734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb86e2d-e79e-4692-bdf3-d9e5542c75d1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:24.587228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb86e2d-e79e-4692-bdf3-d9e5542c75d1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:24.587311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb86e2d-e79e-4692-bdf3-d9e5542c75d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:24.657838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ebb86e2d-e79e-4692-bdf3-d9e5542c75d1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:24.659033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ebb86e2d-e79e-4692-bdf3-d9e5542c75d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:24.760650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:24.763792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 696/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:25.053381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629c5c-ed83-477f-b334-30d1565d20dc None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:25.055812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629c5c-ed83-477f-b334-30d1565d20dc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:25.055812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629c5c-ed83-477f-b334-30d1565d20dc None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:25.057096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 670/1366] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 16:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:25.062972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae143e-4123-4594-abe9-24fc7409221b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:25.063381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae143e-4123-4594-abe9-24fc7409221b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:25.063759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae143e-4123-4594-abe9-24fc7409221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:25.149567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d6ae143e-4123-4594-abe9-24fc7409221b None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:25.150943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d6ae143e-4123-4594-abe9-24fc7409221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:25.276493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 697/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:25.284906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629c5c-ed83-477f-b334-30d1565d20dc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:25.286449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-66629c5c-ed83-477f-b334-30d1565d20dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:25.455985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629c5c-ed83-477f-b334-30d1565d20dc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:25.479359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:25.578423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:25.579080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:25.579290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:25.579563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:25.581192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:25.675772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:25.678044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af2831-c7b9-4913-9058-65eea44160f2 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:25.700239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 671/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:26.267863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9912-6ca2-4079-bac9-b6bcfb974d45 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:26.268451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9912-6ca2-4079-bac9-b6bcfb974d45 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:26.268451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9912-6ca2-4079-bac9-b6bcfb974d45 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:26.271985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 698/1369] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:26.275214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b3bc-939d-4a06-87fc-f0cd6999fd44 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:26.275510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b3bc-939d-4a06-87fc-f0cd6999fd44 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:26.275792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2b3bc-939d-4a06-87fc-f0cd6999fd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:26.309661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:26.349062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-09b2b3bc-939d-4a06-87fc-f0cd6999fd44 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:26.350962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-09b2b3bc-939d-4a06-87fc-f0cd6999fd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:26.456403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 672/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:27.043363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4cd7d-87db-476f-b556-2bc65bfb35e2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.043363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4cd7d-87db-476f-b556-2bc65bfb35e2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.043363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4cd7d-87db-476f-b556-2bc65bfb35e2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.045544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 699/1371] 10.222.0.52 () {54 vars in 964 bytes} [Wed Aug 24 16:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:27.051792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e87fb-a31d-4311-bd80-24aeb9558a72 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.052251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e87fb-a31d-4311-bd80-24aeb9558a72 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.052546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e87fb-a31d-4311-bd80-24aeb9558a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.133501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c11e87fb-a31d-4311-bd80-24aeb9558a72 None None] MFA Rules not processed for user `11df13be0c5543748f98b9ea3d83c3b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:27.134831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c11e87fb-a31d-4311-bd80-24aeb9558a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:27.184674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:27.218674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 673/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Aug 24 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:27.227544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4cd7d-87db-476f-b556-2bc65bfb35e2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:27.229253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6eb4cd7d-87db-476f-b556-2bc65bfb35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:27.285680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:27.342231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4cd7d-87db-476f-b556-2bc65bfb35e2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:27.427953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:27.429285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.429638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.429926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.431610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:27.552991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:27.569973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:27.571729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16c9c4-ec84-4c89-85b4-6c6151ffbe31 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:27.593870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 700/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:27.764319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d26c-ecab-477a-b19f-e6f17865219b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.764654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d26c-ecab-477a-b19f-e6f17865219b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.764822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d26c-ecab-477a-b19f-e6f17865219b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.766809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 674/1374] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:27.771593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab365bd-519f-4127-a2d6-735df15a757d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:27.771884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab365bd-519f-4127-a2d6-735df15a757d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:27.772226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab365bd-519f-4127-a2d6-735df15a757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:27.845602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fab365bd-519f-4127-a2d6-735df15a757d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:27.846486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fab365bd-519f-4127-a2d6-735df15a757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:27.966320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 701/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:28.261649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d26c-ecab-477a-b19f-e6f17865219b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:28.263192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e360d26c-ecab-477a-b19f-e6f17865219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:28.414380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d26c-ecab-477a-b19f-e6f17865219b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:28.467009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:28.530084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:28.530948 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:28.531225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:28.531500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:28.533408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:28.624829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:28.626825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fd59f-8b63-4865-aae4-e6ad5e909caf service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:28.645380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 675/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:30.624547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303ee89-12b3-4802-a4f2-661a2fd19548 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:30.625060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303ee89-12b3-4802-a4f2-661a2fd19548 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:30.625157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303ee89-12b3-4802-a4f2-661a2fd19548 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:30.626453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 702/1377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:30.632631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a591a-34bc-4baf-8c72-6ed2ac260ad2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:30.633086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a591a-34bc-4baf-8c72-6ed2ac260ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:30.633226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a591a-34bc-4baf-8c72-6ed2ac260ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:30.705243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-461a591a-34bc-4baf-8c72-6ed2ac260ad2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:30.706472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-461a591a-34bc-4baf-8c72-6ed2ac260ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:30.713806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:30.818433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 676/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:31.113097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303ee89-12b3-4802-a4f2-661a2fd19548 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:31.114691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0303ee89-12b3-4802-a4f2-661a2fd19548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:31.190260 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:31.308330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303ee89-12b3-4802-a4f2-661a2fd19548 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:31.446730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:31.447461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:31.447822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:31.448188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:31.449896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:31.470420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:31.549731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:31.551433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c511e5-38fe-4249-ac87-4bc5a0c3b775 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:31.571795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 703/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:33.569250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba81d81-d26b-453d-862c-88b4cf4952c3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:33.569769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba81d81-d26b-453d-862c-88b4cf4952c3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:33.569998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba81d81-d26b-453d-862c-88b4cf4952c3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:33.572277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 677/1380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:33.578717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264396-7f2b-47d1-a62b-95b9f143eebc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:33.579149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264396-7f2b-47d1-a62b-95b9f143eebc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:33.579275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264396-7f2b-47d1-a62b-95b9f143eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:33.666964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c9264396-7f2b-47d1-a62b-95b9f143eebc None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:33.668182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9264396-7f2b-47d1-a62b-95b9f143eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:33.768013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:33.775233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 704/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:34.098343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba81d81-d26b-453d-862c-88b4cf4952c3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:34.100740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fba81d81-d26b-453d-862c-88b4cf4952c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:34.265975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:34.278815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba81d81-d26b-453d-862c-88b4cf4952c3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:34.401674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:34.402391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:34.403714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:34.403714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:34.404929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:34.508164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:34.509895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f7701-cb48-4d8f-a7f9-05cc9e4e9b39 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:34.520922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:34.530772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 678/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:36.858335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b7d9e-35b0-49ab-8fcf-490b67d1beb4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:36.858921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b7d9e-35b0-49ab-8fcf-490b67d1beb4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:36.858921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b7d9e-35b0-49ab-8fcf-490b67d1beb4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:36.860339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 705/1383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:36.867783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6264d5-c194-4e9f-ad2a-1cc5298f6b03 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:36.868257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6264d5-c194-4e9f-ad2a-1cc5298f6b03 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:36.868522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6264d5-c194-4e9f-ad2a-1cc5298f6b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:36.937804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6e6264d5-c194-4e9f-ad2a-1cc5298f6b03 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:36.939006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6e6264d5-c194-4e9f-ad2a-1cc5298f6b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:37.055546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 679/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:37.366322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b7d9e-35b0-49ab-8fcf-490b67d1beb4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:37.368381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-411b7d9e-35b0-49ab-8fcf-490b67d1beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:37.535444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b7d9e-35b0-49ab-8fcf-490b67d1beb4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:37.604042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:37.656626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:37.657671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:37.658050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:37.658407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:37.660484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:37.755492 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:37.757180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6f18c-5c30-4863-a2ac-74caf1651318 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:37.779048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 706/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:32:39.694551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be28a1-f648-4e4f-8421-b6d9487f936f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:39.695323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be28a1-f648-4e4f-8421-b6d9487f936f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:39.696050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be28a1-f648-4e4f-8421-b6d9487f936f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:39.697909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 680/1386] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:39.705335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024d1e-59fa-4258-93ad-46c190ba9865 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:39.705818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024d1e-59fa-4258-93ad-46c190ba9865 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:39.706060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024d1e-59fa-4258-93ad-46c190ba9865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:39.783723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-31024d1e-59fa-4258-93ad-46c190ba9865 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:39.784986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-31024d1e-59fa-4258-93ad-46c190ba9865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:39.819141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:39.896071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 707/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:40.276483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be28a1-f648-4e4f-8421-b6d9487f936f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:40.278181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-41be28a1-f648-4e4f-8421-b6d9487f936f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:40.359450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:40.438687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be28a1-f648-4e4f-8421-b6d9487f936f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:40.585461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:40.586138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:40.586530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:40.586804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:40.588999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:40.635214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:40.692034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:40.693844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765213dd-3660-445f-8b07-46fc268f3d2e service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:40.713428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 681/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:42.694627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17443120-cb8f-4c4f-a2d0-767fea1b4004 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:42.695197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17443120-cb8f-4c4f-a2d0-767fea1b4004 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:42.695197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17443120-cb8f-4c4f-a2d0-767fea1b4004 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:42.696582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 708/1389] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:42.702015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:42.702425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:42.702560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:42.770050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:42.771314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:42.833451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14314-26de-4198-86bf-743126c6c7e2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:42.834256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14314-26de-4198-86bf-743126c6c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:42.834686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14314-26de-4198-86bf-743126c6c7e2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:42.836394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 709/1390] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:42.841277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef379b-c0ab-4798-9aea-b58d48cedf9d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:42.841560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef379b-c0ab-4798-9aea-b58d48cedf9d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:42.841834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef379b-c0ab-4798-9aea-b58d48cedf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:42.873229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 682/1391] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:42.913988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-98ef379b-c0ab-4798-9aea-b58d48cedf9d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:42.915189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-98ef379b-c0ab-4798-9aea-b58d48cedf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:43.022039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 710/1392] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:32:43.167481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:43.168741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:43.182554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:43.289114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c523d-a248-48f1-9fcf-d61bea3a7ed4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:43.341447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b38b6a-ca48-4dae-a669-4d8ff2a18905 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:43.341741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b38b6a-ca48-4dae-a669-4d8ff2a18905 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:43.342019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b38b6a-ca48-4dae-a669-4d8ff2a18905 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:43.343399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 711/1393] 10.222.0.52 () {54 vars in 967 bytes} [Wed Aug 24 16:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:32:43.348242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:43.348526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:43.348806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:43.364359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:43.410323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:43.410937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:43.411217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:43.411490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:43.413541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:43.422500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:43.428059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] MFA Rules not processed for user `3671de0f76cc475db615dcc9b351c271`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:43.429299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:43.506574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:43.508334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ba213-af5b-4050-a6fd-77e289cfe239 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:43.508703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 712/1394] 10.222.0.52 () {58 vars in 1064 bytes} [Wed Aug 24 16:32:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:32:43.516330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:43.525895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da94ada-821d-48d1-adf1-40bfefc32f98 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:43.527458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 683/1395] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:43.608804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:43.609431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:43.609713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:43.609989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:43.611771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:43.647015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:43.734654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:43.736147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df12def-48f5-4e1e-bd1a-3351a6182ad7 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:43.755001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 713/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:32:44.501556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca899f6a-4373-446e-aa37-152dd873cc04 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:44.502017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca899f6a-4373-446e-aa37-152dd873cc04 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:44.502109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca899f6a-4373-446e-aa37-152dd873cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:44.569821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ca899f6a-4373-446e-aa37-152dd873cc04 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:44.571038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ca899f6a-4373-446e-aa37-152dd873cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:44.656912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:44.665802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 684/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:44.708060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd125b6-e71b-41e5-b3f1-115c4d639a01 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:44.708348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd125b6-e71b-41e5-b3f1-115c4d639a01 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:44.708646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd125b6-e71b-41e5-b3f1-115c4d639a01 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:44.710222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 714/1398] 10.222.0.52 () {54 vars in 943 bytes} [Wed Aug 24 16:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:44.716885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fd898-f5ff-443e-a8e2-a0d1a7de03e4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:44.717175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fd898-f5ff-443e-a8e2-a0d1a7de03e4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:44.717449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fd898-f5ff-443e-a8e2-a0d1a7de03e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:44.794545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e18fd898-f5ff-443e-a8e2-a0d1a7de03e4 None None] MFA Rules not processed for user `d1e90301e20d4d4ea7f11c7bef1ef4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:44.795664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e18fd898-f5ff-443e-a8e2-a0d1a7de03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:44.903026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 685/1399] 10.222.0.52 () {58 vars in 1040 bytes} [Wed Aug 24 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:32:44.913325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd125b6-e71b-41e5-b3f1-115c4d639a01 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:44.914936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2dd125b6-e71b-41e5-b3f1-115c4d639a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:45.058562 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:45.061464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd125b6-e71b-41e5-b3f1-115c4d639a01 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:45.167040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:45.168109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:45.168579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:45.168927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:45.170596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:45.254559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:45.256847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aacc9a-1b4a-4f92-bf31-92952db50eaf service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:45.265006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 715/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:32:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:32:45.493060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16f949-1c14-46eb-b375-f1aa2032e63a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:45.493395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16f949-1c14-46eb-b375-f1aa2032e63a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:45.493624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16f949-1c14-46eb-b375-f1aa2032e63a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:45.494992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 686/1401] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:45.500266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779dcbb-ac3d-45d3-a5e1-fd815dc28641 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:45.500561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779dcbb-ac3d-45d3-a5e1-fd815dc28641 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:45.500849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779dcbb-ac3d-45d3-a5e1-fd815dc28641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:45.544617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:45.583840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5779dcbb-ac3d-45d3-a5e1-fd815dc28641 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:45.585289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5779dcbb-ac3d-45d3-a5e1-fd815dc28641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:45.718429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 716/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:46.013245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16f949-1c14-46eb-b375-f1aa2032e63a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:46.014789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7d16f949-1c14-46eb-b375-f1aa2032e63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:46.086617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:46.187013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16f949-1c14-46eb-b375-f1aa2032e63a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:46.322282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:46.322811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:46.323089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:46.323357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:46.325042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:46.344849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:46.432589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:46.434626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2a278-2217-46b2-8c67-05863b758513 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:46.456773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 687/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:48.459903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405be7-3740-4dcb-b6bb-652f49aba43c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:48.459903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405be7-3740-4dcb-b6bb-652f49aba43c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:48.459903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405be7-3740-4dcb-b6bb-652f49aba43c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:48.464344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 717/1404] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:48.466635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a545bf-3110-4cc1-a18d-c446cd0a1abc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:48.467171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a545bf-3110-4cc1-a18d-c446cd0a1abc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:48.467361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a545bf-3110-4cc1-a18d-c446cd0a1abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:48.571737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-26a545bf-3110-4cc1-a18d-c446cd0a1abc None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:48.573000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-26a545bf-3110-4cc1-a18d-c446cd0a1abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:48.689234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:48.696120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 688/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:48.988173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405be7-3740-4dcb-b6bb-652f49aba43c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:48.994462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-19405be7-3740-4dcb-b6bb-652f49aba43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:49.109707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:49.160114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405be7-3740-4dcb-b6bb-652f49aba43c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:49.292163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:49.292707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:49.292989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:49.293259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:49.298380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:49.375836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:49.397823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:49.399429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a2d53-9a57-464a-824a-d683def82a1d service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:49.421036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 718/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:32:51.410599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa199e-a165-483d-b6f2-61d4d42213da None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:51.411263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa199e-a165-483d-b6f2-61d4d42213da None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:51.411263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa199e-a165-483d-b6f2-61d4d42213da None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:51.413103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 689/1407] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:32:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:32:51.419330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc308cf-f0b7-47ed-8fce-2782968bb9bd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:51.419891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc308cf-f0b7-47ed-8fce-2782968bb9bd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:51.422082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc308cf-f0b7-47ed-8fce-2782968bb9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:51.499966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3dc308cf-f0b7-47ed-8fce-2782968bb9bd None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:32:51.501331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3dc308cf-f0b7-47ed-8fce-2782968bb9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:51.622895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 719/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:32:51.924297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa199e-a165-483d-b6f2-61d4d42213da None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:32:51.925930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2caa199e-a165-483d-b6f2-61d4d42213da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:52.098031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa199e-a165-483d-b6f2-61d4d42213da None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:32:52.170864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:32:52.221508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:32:52.222109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:32:52.222386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:32:52.222653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:32:52.224646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:32:52.322545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:32:52.324512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6bd52-23b9-4297-be3d-92754af73594 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:32:52.344723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 690/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:01.618795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=79 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:02.342212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=64 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:07.453059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fc85c-9442-4870-ac60-ef111de21217 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:07.453671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fc85c-9442-4870-ac60-ef111de21217 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:07.453671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fc85c-9442-4870-ac60-ef111de21217 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:07.454950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 720/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Aug 24 16:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:07.460233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1696aa7-e526-4aeb-948a-d45b1d53b326 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:07.460707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1696aa7-e526-4aeb-948a-d45b1d53b326 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:07.460810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1696aa7-e526-4aeb-948a-d45b1d53b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:07.539037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b1696aa7-e526-4aeb-948a-d45b1d53b326 None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:07.540331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b1696aa7-e526-4aeb-948a-d45b1d53b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:07.647527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 691/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 24 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:07.662622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aa4d4-654f-4f9d-a7c5-3a27ca5a40c2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:07.662900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aa4d4-654f-4f9d-a7c5-3a27ca5a40c2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:07.663176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aa4d4-654f-4f9d-a7c5-3a27ca5a40c2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:07.664542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 721/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Aug 24 16:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:07.669085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ed937-ba4c-47da-bfb2-efdaac18eb68 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:07.669362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ed937-ba4c-47da-bfb2-efdaac18eb68 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:07.669631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ed937-ba4c-47da-bfb2-efdaac18eb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:07.741150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:07.744605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bf3ed937-ba4c-47da-bfb2-efdaac18eb68 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:07.746210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bf3ed937-ba4c-47da-bfb2-efdaac18eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:07.859673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 692/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 24 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:33:10.263240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265c15-1f9e-456e-bed9-f4feddf987e4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:10.263788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265c15-1f9e-456e-bed9-f4feddf987e4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:10.263788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265c15-1f9e-456e-bed9-f4feddf987e4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:10.265110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 722/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:10.271096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d0796-2a5d-485c-8756-5e2d7ad38379 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:10.271596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d0796-2a5d-485c-8756-5e2d7ad38379 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:10.271702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d0796-2a5d-485c-8756-5e2d7ad38379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:10.338633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4d8d0796-2a5d-485c-8756-5e2d7ad38379 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:10.339843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4d8d0796-2a5d-485c-8756-5e2d7ad38379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:10.403220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:10.440856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 693/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:10.737889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265c15-1f9e-456e-bed9-f4feddf987e4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:10.740198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6f265c15-1f9e-456e-bed9-f4feddf987e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:10.885286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265c15-1f9e-456e-bed9-f4feddf987e4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:10.956840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:11.003154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:11.003691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:11.004002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:11.004285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:11.006698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:11.095866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:11.097480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a611d8-fdc9-4dfa-9fd6-ec157535648a service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:11.116728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 723/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:12.015562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50a6a0-5ef6-488b-936a-f5e83cf4a95b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:12.016177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50a6a0-5ef6-488b-936a-f5e83cf4a95b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:12.016177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50a6a0-5ef6-488b-936a-f5e83cf4a95b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:12.017495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 694/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:12.022893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f153c9-7c17-46d4-b676-9b0e3fbe151e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:12.023334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f153c9-7c17-46d4-b676-9b0e3fbe151e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:12.023438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f153c9-7c17-46d4-b676-9b0e3fbe151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:12.099528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-69f153c9-7c17-46d4-b676-9b0e3fbe151e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:12.100761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-69f153c9-7c17-46d4-b676-9b0e3fbe151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:12.127630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:12.198325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 724/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:12.353735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50a6a0-5ef6-488b-936a-f5e83cf4a95b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:12.355274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9a50a6a0-5ef6-488b-936a-f5e83cf4a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:12.502428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50a6a0-5ef6-488b-936a-f5e83cf4a95b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:12.543350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:12.618929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:12.619465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:12.619745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:12.620273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:12.621756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:12.713343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:12.714933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aec2d-5b85-438c-a511-28609b2faab7 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:12.733630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 695/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:13.622233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c4f2f-2a87-46bf-b2be-3ecf41df3bf8 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:13.622842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c4f2f-2a87-46bf-b2be-3ecf41df3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:13.622842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c4f2f-2a87-46bf-b2be-3ecf41df3bf8 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:13.624154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 725/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:13.629090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95279ea-0c91-48f8-bfc7-30fc3ffa95f7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:13.630096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95279ea-0c91-48f8-bfc7-30fc3ffa95f7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:13.630674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95279ea-0c91-48f8-bfc7-30fc3ffa95f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:13.684181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:13.706445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c95279ea-0c91-48f8-bfc7-30fc3ffa95f7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:13.707665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c95279ea-0c91-48f8-bfc7-30fc3ffa95f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:13.802427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 696/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:13.960817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c4f2f-2a87-46bf-b2be-3ecf41df3bf8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:13.962259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-216c4f2f-2a87-46bf-b2be-3ecf41df3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:14.114417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c4f2f-2a87-46bf-b2be-3ecf41df3bf8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:14.127214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:14.229267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:14.229793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:14.230073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:14.230346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:14.231939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:14.329390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:14.330984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c537f1e-5a0e-4904-8eb1-3560957e8064 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:14.349493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 726/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:15.229994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe55ef-528b-4529-a358-a1c95152ddca None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:15.230545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe55ef-528b-4529-a358-a1c95152ddca None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:15.230545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe55ef-528b-4529-a358-a1c95152ddca None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:15.231904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 697/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:15.236912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813c36b-3e1c-41d8-b525-a2882f6ef1ec None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:15.237354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813c36b-3e1c-41d8-b525-a2882f6ef1ec None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:15.237465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813c36b-3e1c-41d8-b525-a2882f6ef1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:15.258123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:15.319241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5813c36b-3e1c-41d8-b525-a2882f6ef1ec None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:15.320412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5813c36b-3e1c-41d8-b525-a2882f6ef1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:15.416293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 727/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:15.569006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe55ef-528b-4529-a358-a1c95152ddca None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:15.570636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8dbe55ef-528b-4529-a358-a1c95152ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:15.681283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:15.717904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe55ef-528b-4529-a358-a1c95152ddca None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:15.829258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:15.829769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:15.830050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:15.830322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:15.831933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:15.907279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:15.922020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:15.923611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef4cfa-c826-4156-b1c9-506510765a29 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:15.942555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 698/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:16.807682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032bc70-e1bc-4a89-832f-0b627050c94f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:16.808292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032bc70-e1bc-4a89-832f-0b627050c94f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:16.808292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032bc70-e1bc-4a89-832f-0b627050c94f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:16.809665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 728/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:16.814719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2e6e-9160-47c1-a687-e7f9c04b9851 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:16.815152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2e6e-9160-47c1-a687-e7f9c04b9851 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:16.815266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2e6e-9160-47c1-a687-e7f9c04b9851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:16.892128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ce6e2e6e-9160-47c1-a687-e7f9c04b9851 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:16.893406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ce6e2e6e-9160-47c1-a687-e7f9c04b9851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:16.991140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 699/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:17.143584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032bc70-e1bc-4a89-832f-0b627050c94f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:17.145185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7032bc70-e1bc-4a89-832f-0b627050c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:17.229514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:17.294358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032bc70-e1bc-4a89-832f-0b627050c94f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:17.409026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:17.409549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:17.409827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:17.410102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:17.412322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:17.460523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:17.502868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:17.504498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217b9f6-fd7b-4f47-a900-ce1085c66443 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:17.522524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 729/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:18.345533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bfa19-b45a-406a-9e38-f73df4e69671 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:18.346064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bfa19-b45a-406a-9e38-f73df4e69671 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:18.346064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bfa19-b45a-406a-9e38-f73df4e69671 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:18.347389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 700/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:18.353016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da085-7c5b-4ef7-ae97-d08e4741964a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:18.353426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da085-7c5b-4ef7-ae97-d08e4741964a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:18.353569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da085-7c5b-4ef7-ae97-d08e4741964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:18.430425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-dd3da085-7c5b-4ef7-ae97-d08e4741964a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:18.431648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dd3da085-7c5b-4ef7-ae97-d08e4741964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:18.527639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 730/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:18.679387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bfa19-b45a-406a-9e38-f73df4e69671 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:18.680963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-db5bfa19-b45a-406a-9e38-f73df4e69671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:18.730514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:18.828582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bfa19-b45a-406a-9e38-f73df4e69671 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:18.945774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:18.946286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:18.946560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:18.946825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:18.948441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:18.955194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:19.038260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:19.039871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d23d7-2720-41a4-a96e-b12370abae20 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:19.067715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 701/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:19.915161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7318b3-63f3-4131-8b26-c83e3686c0f3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:19.915730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7318b3-63f3-4131-8b26-c83e3686c0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:19.915730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7318b3-63f3-4131-8b26-c83e3686c0f3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:19.917178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 731/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:19.922188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96465ff3-f323-43e0-9bf2-192f9aa08155 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:19.922642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96465ff3-f323-43e0-9bf2-192f9aa08155 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:19.922762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96465ff3-f323-43e0-9bf2-192f9aa08155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:20.003040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-96465ff3-f323-43e0-9bf2-192f9aa08155 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:20.004287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-96465ff3-f323-43e0-9bf2-192f9aa08155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:20.074803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:20.103763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 702/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:20.257236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7318b3-63f3-4131-8b26-c83e3686c0f3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:20.258762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ed7318b3-63f3-4131-8b26-c83e3686c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:20.277703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:20.407911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7318b3-63f3-4131-8b26-c83e3686c0f3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:20.499213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:20.525147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:20.525753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:20.526031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:20.526304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:20.527899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:20.617491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:20.619091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df66ca7-9634-4614-85f1-d23db3456385 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:20.636635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 732/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:21.440064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5d35-b08f-410e-bc62-b2a3dca10323 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:21.440716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5d35-b08f-410e-bc62-b2a3dca10323 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:21.440716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5d35-b08f-410e-bc62-b2a3dca10323 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:21.441984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 703/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:21.447929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf262-c49c-40ac-9cef-e36bc95dcbcb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:21.448385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf262-c49c-40ac-9cef-e36bc95dcbcb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:21.448510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf262-c49c-40ac-9cef-e36bc95dcbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:21.526653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-477cf262-c49c-40ac-9cef-e36bc95dcbcb None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:21.527856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-477cf262-c49c-40ac-9cef-e36bc95dcbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:21.570514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:21.628296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 733/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:21.810472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5d35-b08f-410e-bc62-b2a3dca10323 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:21.812033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c15a5d35-b08f-410e-bc62-b2a3dca10323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:21.957454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5d35-b08f-410e-bc62-b2a3dca10323 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:22.012202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:22.071619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:22.072188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:22.072472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:22.072750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:22.074369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:22.168364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:22.169942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a562226-e8d4-4ffd-9b50-72671b4377b3 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:22.188206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 704/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:22.989285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab390d2-853e-4db4-8599-a416b651e6d7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:22.989867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab390d2-853e-4db4-8599-a416b651e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:22.989975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab390d2-853e-4db4-8599-a416b651e6d7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:22.991350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 734/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:22.996297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbf880-082c-479f-ba6c-bb6cd32cdcda None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:22.996763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbf880-082c-479f-ba6c-bb6cd32cdcda None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:22.996873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbf880-082c-479f-ba6c-bb6cd32cdcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:23.074312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-11bbf880-082c-479f-ba6c-bb6cd32cdcda None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:23.075522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-11bbf880-082c-479f-ba6c-bb6cd32cdcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:23.077553 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:23.175273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 705/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:23.327717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab390d2-853e-4db4-8599-a416b651e6d7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:23.329201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1ab390d2-853e-4db4-8599-a416b651e6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:23.478606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab390d2-853e-4db4-8599-a416b651e6d7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:23.504968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:23.590288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:23.590805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:23.591079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:23.591345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:23.592986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:23.680547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:23.682131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef52b1-8959-493e-a4ff-f5462565198e service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:23.700702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 735/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:25.545179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbe32c-27c2-4484-976f-c4acd9b0d957 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:25.545761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbe32c-27c2-4484-976f-c4acd9b0d957 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:25.545761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbe32c-27c2-4484-976f-c4acd9b0d957 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:25.547080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 706/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:25.553042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0a240-a499-4cc0-a254-e986e36b172e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:25.553463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0a240-a499-4cc0-a254-e986e36b172e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:25.553592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0a240-a499-4cc0-a254-e986e36b172e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:25.594125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:25.620145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c1f0a240-a499-4cc0-a254-e986e36b172e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:25.621368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c1f0a240-a499-4cc0-a254-e986e36b172e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:25.717594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 736/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:26.008910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbe32c-27c2-4484-976f-c4acd9b0d957 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:26.010482 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8dfbe32c-27c2-4484-976f-c4acd9b0d957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:26.116586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbe32c-27c2-4484-976f-c4acd9b0d957 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:26.149906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:26.192240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:26.193435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:26.193716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:26.193995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:26.195611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:26.303798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:26.305194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3e65c-1ca5-4f98-961b-570ae6a0aead service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:26.323645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 707/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:28.412682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775faf-e235-4d83-95ec-42a64908b866 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:28.413272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775faf-e235-4d83-95ec-42a64908b866 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:28.413272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775faf-e235-4d83-95ec-42a64908b866 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:28.414632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 737/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:28.420219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52511ee-9eab-4c3a-b81d-8cef3e81fbcd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:28.420651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52511ee-9eab-4c3a-b81d-8cef3e81fbcd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:28.420777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52511ee-9eab-4c3a-b81d-8cef3e81fbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:28.486230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c52511ee-9eab-4c3a-b81d-8cef3e81fbcd None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:28.487437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c52511ee-9eab-4c3a-b81d-8cef3e81fbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:28.500221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:28.585775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 708/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:28.877672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775faf-e235-4d83-95ec-42a64908b866 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:28.879256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c1775faf-e235-4d83-95ec-42a64908b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:28.984207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775faf-e235-4d83-95ec-42a64908b866 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:28.996423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:29.060549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:29.061156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:29.061556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:29.061835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:29.063445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:29.172163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:29.173528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bac20-abb4-4d78-af4f-4831e0d3ef9a service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:29.191845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 738/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:31.588517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624db63-011a-4c88-b9ec-be10b6e98acf None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:31.589114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624db63-011a-4c88-b9ec-be10b6e98acf None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:31.589114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624db63-011a-4c88-b9ec-be10b6e98acf None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:31.590404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 709/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:31.595799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b739a8-d626-4e9f-9ecb-be637cefd315 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:31.596214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b739a8-d626-4e9f-9ecb-be637cefd315 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:31.596400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b739a8-d626-4e9f-9ecb-be637cefd315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:31.645192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:31.664086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-45b739a8-d626-4e9f-9ecb-be637cefd315 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:31.665323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-45b739a8-d626-4e9f-9ecb-be637cefd315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:31.764571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 739/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:32.051525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624db63-011a-4c88-b9ec-be10b6e98acf None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:32.053126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1624db63-011a-4c88-b9ec-be10b6e98acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:32.167802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624db63-011a-4c88-b9ec-be10b6e98acf None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:32.206548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:32.241899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:32.242507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:32.242786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:32.243059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:32.244712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:32.353821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:32.355258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ac02e-d4e3-4c4e-a854-a34b25b609d7 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:32.373640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 710/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:34.459520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7be3-9f15-4974-9277-83b817f950c7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:34.460105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7be3-9f15-4974-9277-83b817f950c7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:34.460105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7be3-9f15-4974-9277-83b817f950c7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:34.461440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 740/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:34.466640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a53a-0e6b-43b0-8a32-10ab6282060d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:34.467091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a53a-0e6b-43b0-8a32-10ab6282060d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:34.467188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a53a-0e6b-43b0-8a32-10ab6282060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:34.539291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-df25a53a-0e6b-43b0-8a32-10ab6282060d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:34.540521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-df25a53a-0e6b-43b0-8a32-10ab6282060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:34.560783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:34.637654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 711/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:34.924716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7be3-9f15-4974-9277-83b817f950c7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:34.926301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-900f7be3-9f15-4974-9277-83b817f950c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:35.030358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7be3-9f15-4974-9277-83b817f950c7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:35.051434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:35.105485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:35.106094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:35.106375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:35.106650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:35.108281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:35.219706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:35.221087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8c044-143f-41ed-ba18-45572aaaaba2 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:35.239450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 741/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:37.201748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b30f6-cd35-4cbc-b06c-7e53989fe16a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:37.202267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b30f6-cd35-4cbc-b06c-7e53989fe16a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:37.202267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b30f6-cd35-4cbc-b06c-7e53989fe16a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:37.203578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 712/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:33:37.209665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b1b5-47e8-4159-92bb-190104051984 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:37.210109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b1b5-47e8-4159-92bb-190104051984 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:37.210208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b1b5-47e8-4159-92bb-190104051984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:37.272563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:37.275319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-59f1b1b5-47e8-4159-92bb-190104051984 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:37.277096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-59f1b1b5-47e8-4159-92bb-190104051984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:37.372319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 742/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:37.664084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b30f6-cd35-4cbc-b06c-7e53989fe16a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:37.665697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d08b30f6-cd35-4cbc-b06c-7e53989fe16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:37.767840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b30f6-cd35-4cbc-b06c-7e53989fe16a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:37.818498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:37.843956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:37.844584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:37.844862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:37.845139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:37.846753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:37.953849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:37.955192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f635f53-e5be-4e8e-a730-9601906e3254 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:37.974178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 713/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:40.056819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9b647-1e0b-41fb-b338-b09d926a398e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:40.057315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9b647-1e0b-41fb-b338-b09d926a398e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:40.057424 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9b647-1e0b-41fb-b338-b09d926a398e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:40.058734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 743/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:40.064167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdccb0-7884-4f07-bc02-2062e9bd10d8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:40.064591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdccb0-7884-4f07-bc02-2062e9bd10d8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:40.064721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdccb0-7884-4f07-bc02-2062e9bd10d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:40.132549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-06cdccb0-7884-4f07-bc02-2062e9bd10d8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:40.133743 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-06cdccb0-7884-4f07-bc02-2062e9bd10d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:40.162962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:40.231908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 714/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:40.519583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9b647-1e0b-41fb-b338-b09d926a398e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:40.521215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bdb9b647-1e0b-41fb-b338-b09d926a398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:40.623406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9b647-1e0b-41fb-b338-b09d926a398e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:40.650489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:40.696708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:40.697312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:40.697592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:40.697866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:40.699472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:40.807691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:40.809081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01b010-4623-4f62-9999-8b3f25f395d1 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:40.826827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 744/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:42.971835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3880-3c1d-4f8b-9ec5-f2bb0b3f9583 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:42.972430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3880-3c1d-4f8b-9ec5-f2bb0b3f9583 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:42.972430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b3880-3c1d-4f8b-9ec5-f2bb0b3f9583 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:42.974840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 715/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:42.980433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a456d1-44b5-4242-8b33-037c674498ff None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:42.980866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a456d1-44b5-4242-8b33-037c674498ff None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:42.980993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a456d1-44b5-4242-8b33-037c674498ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:43.054162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-70a456d1-44b5-4242-8b33-037c674498ff None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:43.055384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-70a456d1-44b5-4242-8b33-037c674498ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:43.063862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:43.152261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 745/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:43.449982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3880-3c1d-4f8b-9ec5-f2bb0b3f9583 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:43.451541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-235b3880-3c1d-4f8b-9ec5-f2bb0b3f9583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:43.555576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b3880-3c1d-4f8b-9ec5-f2bb0b3f9583 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:43.612302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:43.631555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:43.632197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:43.632479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:43.632750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:43.634388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:43.743793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:43.745188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23965f4-e2c3-4b0c-bf2e-7258c1abbcf7 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:43.763637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 716/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:45.741926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9069f5-6f4d-46eb-bd70-b3c5028d2835 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:45.742551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9069f5-6f4d-46eb-bd70-b3c5028d2835 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:45.742551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9069f5-6f4d-46eb-bd70-b3c5028d2835 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:45.743829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 746/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:45.749075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ce1b3-cbe3-431d-9eb3-a89811f66749 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:45.749507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ce1b3-cbe3-431d-9eb3-a89811f66749 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:45.749664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ce1b3-cbe3-431d-9eb3-a89811f66749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:45.815381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5b9ce1b3-cbe3-431d-9eb3-a89811f66749 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:45.816613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5b9ce1b3-cbe3-431d-9eb3-a89811f66749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:45.855296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:45.915506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 717/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:46.195300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9069f5-6f4d-46eb-bd70-b3c5028d2835 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:46.196860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5d9069f5-6f4d-46eb-bd70-b3c5028d2835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:46.298056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9069f5-6f4d-46eb-bd70-b3c5028d2835 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:46.336976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:46.372301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:46.372898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:46.373175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:46.373445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:46.375031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:46.483653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:46.485122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4dd454-a3cc-4124-bc2c-d60fb263d5ea service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:46.503252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 747/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:48.431111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddf2e0-d1a3-4dae-bf1f-8297bba077f2 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:48.431706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddf2e0-d1a3-4dae-bf1f-8297bba077f2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:48.431706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddf2e0-d1a3-4dae-bf1f-8297bba077f2 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:48.433197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 718/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:48.438564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bda137-93ac-4778-8b17-727116fd7c7f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:48.438972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bda137-93ac-4778-8b17-727116fd7c7f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:48.439140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bda137-93ac-4778-8b17-727116fd7c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:48.504995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b3bda137-93ac-4778-8b17-727116fd7c7f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:48.506213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b3bda137-93ac-4778-8b17-727116fd7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:48.524550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:48.606810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 748/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:48.891530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddf2e0-d1a3-4dae-bf1f-8297bba077f2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:48.893135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-54ddf2e0-d1a3-4dae-bf1f-8297bba077f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:48.996505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddf2e0-d1a3-4dae-bf1f-8297bba077f2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:49.063040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:49.071000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:49.071599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:49.071875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:49.072172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:49.073933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:49.181236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:49.182587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45baede-7093-4047-8efd-2385b875897d service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:49.201084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 719/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:33:51.268513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbda7e-a2ae-4a3d-bd23-cd60c8a504cb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:51.269111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbda7e-a2ae-4a3d-bd23-cd60c8a504cb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:51.269111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbda7e-a2ae-4a3d-bd23-cd60c8a504cb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:51.271080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 749/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:33:51.276680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abbff2-8bd1-4ce9-96d6-a0ae11be4af6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:51.277124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abbff2-8bd1-4ce9-96d6-a0ae11be4af6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:51.277234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abbff2-8bd1-4ce9-96d6-a0ae11be4af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:51.344044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d5abbff2-8bd1-4ce9-96d6-a0ae11be4af6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:33:51.345267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d5abbff2-8bd1-4ce9-96d6-a0ae11be4af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:51.392151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:51.441898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 720/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:33:51.735918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbda7e-a2ae-4a3d-bd23-cd60c8a504cb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:33:51.737507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-75dbda7e-a2ae-4a3d-bd23-cd60c8a504cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:51.840292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbda7e-a2ae-4a3d-bd23-cd60c8a504cb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:33:51.887067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:33:51.912937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:33:51.913538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:33:51.913820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:33:51.914094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:33:51.915691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:33:52.023075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:33:52.024531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1905a3-88ca-4fb8-be67-4145d45c12b6 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:33:52.043425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 750/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:01.440016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=22 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:02.041155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=66 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:12.002165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b12b-b4ff-48be-b126-7252b8ec11b7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:12.002733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b12b-b4ff-48be-b126-7252b8ec11b7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:12.002733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b12b-b4ff-48be-b126-7252b8ec11b7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:12.004098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 721/1471] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:12.010762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88074-89d4-410a-aabd-dba003c70a0f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:12.011183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88074-89d4-410a-aabd-dba003c70a0f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:12.011411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac88074-89d4-410a-aabd-dba003c70a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:12.077978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ac88074-89d4-410a-aabd-dba003c70a0f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:12.079259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ac88074-89d4-410a-aabd-dba003c70a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:12.186035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 751/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:12.480884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555e7a6-1da6-49a9-9736-e903f56c2cc6 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:12.481164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555e7a6-1da6-49a9-9736-e903f56c2cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:12.481436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555e7a6-1da6-49a9-9736-e903f56c2cc6 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:12.482764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 722/1473] 10.222.0.52 () {54 vars in 967 bytes} [Wed Aug 24 16:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:12.487482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d18d6-8bc0-4f5a-aff3-00fd139dee21 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:12.487774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d18d6-8bc0-4f5a-aff3-00fd139dee21 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:12.488086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d18d6-8bc0-4f5a-aff3-00fd139dee21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:12.564627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-987d18d6-8bc0-4f5a-aff3-00fd139dee21 None None] MFA Rules not processed for user `3671de0f76cc475db615dcc9b351c271`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:12.565839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-987d18d6-8bc0-4f5a-aff3-00fd139dee21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:12.587044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:12.642965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 752/1474] 10.222.0.52 () {58 vars in 1064 bytes} [Wed Aug 24 16:34:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:12.650806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555e7a6-1da6-49a9-9736-e903f56c2cc6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:12.652410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0555e7a6-1da6-49a9-9736-e903f56c2cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:12.758148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555e7a6-1da6-49a9-9736-e903f56c2cc6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:12.829511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:12.830111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:12.830387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:12.830657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:12.832273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:12.885460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:12.940466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:12.941894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b3981-0d19-4371-8ea9-1ff271a7164e service glance] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:12.959811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 723/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:14.943239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae0dea-529c-47f3-bc2e-a3d3fba1731f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:14.943785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae0dea-529c-47f3-bc2e-a3d3fba1731f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:14.943785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae0dea-529c-47f3-bc2e-a3d3fba1731f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:14.945154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 753/1476] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:14.950520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca8e0a-2591-4a81-8510-1b0f70ff12a1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:14.950933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca8e0a-2591-4a81-8510-1b0f70ff12a1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:14.951087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca8e0a-2591-4a81-8510-1b0f70ff12a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:15.017267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0eca8e0a-2591-4a81-8510-1b0f70ff12a1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:15.018468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0eca8e0a-2591-4a81-8510-1b0f70ff12a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:15.116026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 724/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:15.396258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae0dea-529c-47f3-bc2e-a3d3fba1731f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:15.397807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4fae0dea-529c-47f3-bc2e-a3d3fba1731f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:15.524262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae0dea-529c-47f3-bc2e-a3d3fba1731f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:15.572815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:15.617491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:15.618092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:15.618371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:15.618640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:15.619277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:15.621078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:34:15.621078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:34:15.621078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:34:15.622228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf26c0-c943-4c20-967c-1792dc7d864d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:15.641270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 754/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 16:34:15 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:34:17.547398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac293-e29e-40d3-849e-62c456463df5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:17.547995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac293-e29e-40d3-849e-62c456463df5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:17.547995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac293-e29e-40d3-849e-62c456463df5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:17.549350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 725/1479] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:17.555851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55bad3-82a2-48a3-8168-1320bbfa2469 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:17.556317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55bad3-82a2-48a3-8168-1320bbfa2469 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:17.556422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55bad3-82a2-48a3-8168-1320bbfa2469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:17.625342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4c55bad3-82a2-48a3-8168-1320bbfa2469 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:17.626541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4c55bad3-82a2-48a3-8168-1320bbfa2469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:17.727747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 755/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:18.020526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac293-e29e-40d3-849e-62c456463df5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:18.022098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4c3ac293-e29e-40d3-849e-62c456463df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:18.035390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:18.177654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac293-e29e-40d3-849e-62c456463df5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:18.257545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:18.291855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:18.292489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:18.292776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:18.293051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:18.294662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:18.382993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:18.384611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4de23e-2300-4eb9-b34d-d1e93d1461db service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:18.402966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 726/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:20.499695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae231e5-aff6-48ca-83b7-004bf231ae3d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:20.500263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae231e5-aff6-48ca-83b7-004bf231ae3d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:20.500263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae231e5-aff6-48ca-83b7-004bf231ae3d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:20.501606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 756/1482] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:20.507843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b164e-f7e0-445b-bc39-46bc4af15a0d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:20.508308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b164e-f7e0-445b-bc39-46bc4af15a0d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:20.508420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b164e-f7e0-445b-bc39-46bc4af15a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:20.574923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-546b164e-f7e0-445b-bc39-46bc4af15a0d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:20.576170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-546b164e-f7e0-445b-bc39-46bc4af15a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:20.624343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:20.676501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 727/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:20.964837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae231e5-aff6-48ca-83b7-004bf231ae3d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:20.966473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5ae231e5-aff6-48ca-83b7-004bf231ae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:20.989372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:21.186028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae231e5-aff6-48ca-83b7-004bf231ae3d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:21.258861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:21.259384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:21.259657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:21.259921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:21.261528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:21.334405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:21.369245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:21.370658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc47649-54c3-4b49-96fc-4255b80269a7 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:21.388046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 757/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:23.396876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8ef2-2ba0-42c1-9edd-05d86c6547a7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:23.397424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8ef2-2ba0-42c1-9edd-05d86c6547a7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:23.397424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8ef2-2ba0-42c1-9edd-05d86c6547a7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:23.398788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 728/1485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:23.404185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55cf17-90f7-4caa-b176-dc40f06fec64 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:23.404635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55cf17-90f7-4caa-b176-dc40f06fec64 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:23.404814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55cf17-90f7-4caa-b176-dc40f06fec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:23.471085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0c55cf17-90f7-4caa-b176-dc40f06fec64 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:23.472379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0c55cf17-90f7-4caa-b176-dc40f06fec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:23.570139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 758/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:23.858565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c8ef2-2ba0-42c1-9edd-05d86c6547a7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:23.860188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e25c8ef2-2ba0-42c1-9edd-05d86c6547a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:23.963368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c8ef2-2ba0-42c1-9edd-05d86c6547a7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:24.037589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:24.039755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:24.040611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:24.041093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:24.041540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:24.043162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:24.163383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:24.164761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e220d-c3e2-4fda-8449-926373786393 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:24.183745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 729/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:26.191086 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e8879-5636-41ab-b0cd-cf0006162f95 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:26.191650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e8879-5636-41ab-b0cd-cf0006162f95 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:26.191650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e8879-5636-41ab-b0cd-cf0006162f95 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:26.193055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 759/1488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:26.199649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33b2d2-969d-4a49-b3c7-96fa137906c1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:26.200071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33b2d2-969d-4a49-b3c7-96fa137906c1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:26.200235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33b2d2-969d-4a49-b3c7-96fa137906c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:26.265774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6b33b2d2-969d-4a49-b3c7-96fa137906c1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:26.266985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6b33b2d2-969d-4a49-b3c7-96fa137906c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:26.320145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:26.366130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 730/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:26.655311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e8879-5636-41ab-b0cd-cf0006162f95 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:26.656885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-182e8879-5636-41ab-b0cd-cf0006162f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:26.693823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:26.763446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e8879-5636-41ab-b0cd-cf0006162f95 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:26.836828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:26.837434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:26.837718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:26.837997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:26.839629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:26.918575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:26.949033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:26.950450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29844ec5-5bfe-409d-b37f-7f7d145cfeb3 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:26.969336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 760/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:29.099150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51940e9-8492-4175-8219-e6b98b891bb0 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:29.099719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51940e9-8492-4175-8219-e6b98b891bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:29.099719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51940e9-8492-4175-8219-e6b98b891bb0 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:29.101195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 731/1491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 16:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:34:29.106387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db314ce-d8a1-4b95-9c08-20977493a75f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:29.106831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db314ce-d8a1-4b95-9c08-20977493a75f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:29.106944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db314ce-d8a1-4b95-9c08-20977493a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:29.176136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5db314ce-d8a1-4b95-9c08-20977493a75f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:34:29.177540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5db314ce-d8a1-4b95-9c08-20977493a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:29.272230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 761/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:34:29.572493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51940e9-8492-4175-8219-e6b98b891bb0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:34:29.574045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a51940e9-8492-4175-8219-e6b98b891bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:29.678276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51940e9-8492-4175-8219-e6b98b891bb0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:34:29.753909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:34:29.754552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:34:29.754887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:34:29.755157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:34:29.756987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:34:29.766222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:29.866626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:34:29.868095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92396f5a-b5d2-40d2-8de4-924d1f343d6b service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:34:29.886103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 732/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:34:39.269745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=82 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:34:39.883765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=50 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:42.192091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:42.192736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:42.192736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:42.266661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:35:42.267858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:42.360722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 762/1494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:35:42.370386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:42.371060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:42.371373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:42.451794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:35:42.453016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:42.548587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 733/1495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:35:42.553445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:42.554646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:42.565890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1bfb5-d159-46e3-92ee-cf59a0083dea None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:42.625790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:42.660272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:42.660875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:42.661152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:42.661420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:42.662052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:35:42.662359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:42.663360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:42.663360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:35:42.663360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:35:42.664269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:42.677661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 763/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:35:42.686772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:42.696688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ce03d-3d68-4d36-b1e7-195796413959 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:42.758206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:42.793757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:42.794363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:42.794643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:42.794918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:42.795555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:42.797354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:42.797354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:35:42.797354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:35:42.798551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:42.855467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 734/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:35:42 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:35:42.859728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:42.861327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:42.896076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4ca66-77a7-4a09-a10b-81484a5987b7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:42.989030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:42.989639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:42.989926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:42.990225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:42.990860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:42.991864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:42.991864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:35:42.991864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:35:42.993102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:43.035293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 764/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:35:42 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:35:43.039594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:43.041183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:43.104016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecf0c6-ca2b-4c66-8adb-503330cc5d0b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:43.187401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:43.199449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:43.200092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:43.200376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:43.200649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:43.201769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:43.202299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:43.202299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:35:43.202299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:35:43.203187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:43.215780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 735/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:35:43.219756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:43.221344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:43.233226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefb007-7262-4058-b743-75c5f7c2b2ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:43.258427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:43.329893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:43.330501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:43.330782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:43.331056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:43.331766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:43.332819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:43.332819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:35:43.332819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:35:43.333764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3afaf-7fd9-42ca-9505-5344f8abe549 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:43.346507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 765/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:35:43.351240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:43.352840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:43.387016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9b45d-d2f8-44b6-bb77-955f3baf53a4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:43.482363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:43.482950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:43.483232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:43.483510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] PATH_INFO: `/v3/projects/f37858604b7b447d82a6b25471cb5371/users/4a8d8d2c187a4bc9b4d0715c53a1e3be/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:43.505649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37858604b7b447d82a6b25471cb5371, user_id=4a8d8d2c187a4bc9b4d0715c53a1e3be, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:43.506681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:43.506681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:35:43.506681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:35:43.508645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e969f-f19c-4379-a117-b035efec5ae8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:43.531847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 736/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:35:43 2022] PUT /identity/v3/projects/f37858604b7b447d82a6b25471cb5371/users/4a8d8d2c187a4bc9b4d0715c53a1e3be/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:35:43.545139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284c15-49e3-4e95-b875-bd8f46bf2a09 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:43.545410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284c15-49e3-4e95-b875-bd8f46bf2a09 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:43.545764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284c15-49e3-4e95-b875-bd8f46bf2a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:43.604588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ba284c15-49e3-4e95-b875-bd8f46bf2a09 None None] MFA Rules not processed for user `4a8d8d2c187a4bc9b4d0715c53a1e3be`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:35:43.605779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ba284c15-49e3-4e95-b875-bd8f46bf2a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:43.681709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 766/1502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:35:43.695485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd981ce-4569-4030-9d1a-0ba79c626b00 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:43.696057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd981ce-4569-4030-9d1a-0ba79c626b00 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:43.696558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd981ce-4569-4030-9d1a-0ba79c626b00 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:43.698168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 737/1503] 10.222.0.52 () {54 vars in 964 bytes} [Wed Aug 24 16:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:35:43.703013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af9e2b-5c2c-4e42-ab05-06fe53ebc61a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:43.703302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af9e2b-5c2c-4e42-ab05-06fe53ebc61a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:43.703604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af9e2b-5c2c-4e42-ab05-06fe53ebc61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:43.736431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:43.780549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f2af9e2b-5c2c-4e42-ab05-06fe53ebc61a None None] MFA Rules not processed for user `11df13be0c5543748f98b9ea3d83c3b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:35:43.781765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f2af9e2b-5c2c-4e42-ab05-06fe53ebc61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:43.857628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 767/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Aug 24 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:35:43.865665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd981ce-4569-4030-9d1a-0ba79c626b00 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:43.867205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-afd981ce-4569-4030-9d1a-0ba79c626b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:43.917745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:43.973244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd981ce-4569-4030-9d1a-0ba79c626b00 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:44.062104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:44.062715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:44.062990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:44.063259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:44.065021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:44.286719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:44.288069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:44.291487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:44.306152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 738/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:35:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:35:44.352217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e74333-ce22-4dfb-afa7-6d8588da1d6a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:44.352650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e74333-ce22-4dfb-afa7-6d8588da1d6a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:44.352776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e74333-ce22-4dfb-afa7-6d8588da1d6a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:44.354104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 768/1506] 10.222.0.52 () {52 vars in 830 bytes} [Wed Aug 24 16:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:35:44.357614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:44.358809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:44.466113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b1ef-510b-4d97-807c-c7da8c8925db service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:44.560266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:44.560817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:44.561093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:44.561363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] PATH_INFO: `/v3/users/4a8d8d2c187a4bc9b4d0715c53a1e3be` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:44.574926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8d8d2c187a4bc9b4d0715c53a1e3be)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:44.575971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:44.575971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:35:44.575971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:35:44.577188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:44.587095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:44.643181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8d8d2c187a4bc9b4d0715c53a1e3be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:44.653459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8d8d2c187a4bc9b4d0715c53a1e3be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:44.678275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8d8d2c187a4bc9b4d0715c53a1e3be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:44.678818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8d8d2c187a4bc9b4d0715c53a1e3be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:44.693289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8d8d2c187a4bc9b4d0715c53a1e3be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:44.698733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 739/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:35:44 2022] DELETE /identity/v3/users/4a8d8d2c187a4bc9b4d0715c53a1e3be => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:35:44.703131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:44.703423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:44.703704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:44.760383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:35:44.761596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:44.837453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:44.851006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 769/1508] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:35:44.861090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:44.862310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:44.963384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463ef3d-d68d-4af0-a82d-655786bd833d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:45.017839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:45.040235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:45.040850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:45.041128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:45.041398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:45.042994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:45.152684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:45.154116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6d690-8831-4cbc-afcf-824678854068 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:45.172955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 740/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:35:45.548650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:35:45.549903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:35:45.561599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cea08-0bea-4966-b484-2ed91a216498 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:35:45.657737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:35:45.658373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:35:45.658655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:35:45.658931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] PATH_INFO: `/v3/projects/f37858604b7b447d82a6b25471cb5371` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:35:45.669530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37858604b7b447d82a6b25471cb5371)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:35:45.670488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:35:45.670488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:35:45.670488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:35:45.671675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:35:45.738069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37858604b7b447d82a6b25471cb5371', 'request_id': 'req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:45.751113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37858604b7b447d82a6b25471cb5371', 'request_id': 'req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:45.760521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37858604b7b447d82a6b25471cb5371', 'request_id': 'req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:45.783001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] The token cache is being invalidate because project f37858604b7b447d82a6b25471cb5371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:35:45.783287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:45.783720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5b26ea0b-d9db-462b-b0af-5c0d28b29c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:35:45.785229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 770/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:35:45 2022] DELETE /identity/v3/projects/f37858604b7b447d82a6b25471cb5371 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:35:55.170687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=62 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:35:55.780799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:14.791763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a55bd2-560d-4e47-a70a-35799225b6f4 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:14.792401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a55bd2-560d-4e47-a70a-35799225b6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:14.792505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a55bd2-560d-4e47-a70a-35799225b6f4 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:14.793841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 741/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:14.798582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9398faa-a75e-4fac-9531-080591750d5e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:14.799061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9398faa-a75e-4fac-9531-080591750d5e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:14.799182 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9398faa-a75e-4fac-9531-080591750d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:14.890217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e9398faa-a75e-4fac-9531-080591750d5e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:14.891498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e9398faa-a75e-4fac-9531-080591750d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:14.996886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 771/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:15.155918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a55bd2-560d-4e47-a70a-35799225b6f4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:15.158801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-75a55bd2-560d-4e47-a70a-35799225b6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:15.279016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a55bd2-560d-4e47-a70a-35799225b6f4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:15.284710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d8911-8028-4ece-9240-453ccc165764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:15.285303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d8911-8028-4ece-9240-453ccc165764 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:15.285577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d8911-8028-4ece-9240-453ccc165764 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:15.286450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d8911-8028-4ece-9240-453ccc165764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:15.294475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d8911-8028-4ece-9240-453ccc165764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:15.296053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d8911-8028-4ece-9240-453ccc165764 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:15.305765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 742/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:16.144620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b5489-a643-4976-821a-b665181be4e3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:16.146730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b5489-a643-4976-821a-b665181be4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:16.146730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b5489-a643-4976-821a-b665181be4e3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:16.147087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 772/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:16.152135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156d2b-ad6d-431b-8a10-ff8d5ef2388e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:16.152614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156d2b-ad6d-431b-8a10-ff8d5ef2388e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:16.152726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156d2b-ad6d-431b-8a10-ff8d5ef2388e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:16.172323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b5156d2b-ad6d-431b-8a10-ff8d5ef2388e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:16.173535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b5156d2b-ad6d-431b-8a10-ff8d5ef2388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:16.181393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 743/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:37:16.337630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b5489-a643-4976-821a-b665181be4e3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:16.353357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b5489-a643-4976-821a-b665181be4e3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:16.358755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aac08-60f0-412e-81cb-0d4deffb0870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:16.359355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aac08-60f0-412e-81cb-0d4deffb0870 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:16.359634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aac08-60f0-412e-81cb-0d4deffb0870 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:16.359906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aac08-60f0-412e-81cb-0d4deffb0870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:16.368911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1aac08-60f0-412e-81cb-0d4deffb0870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:16.370480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1aac08-60f0-412e-81cb-0d4deffb0870 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:16.381160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 773/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:17.021071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd70c-0a13-4a79-b22e-23d89c9d8c17 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:17.021491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd70c-0a13-4a79-b22e-23d89c9d8c17 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:17.021764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd70c-0a13-4a79-b22e-23d89c9d8c17 None None] PATH_INFO: `/v3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:17.023049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 744/1517] 10.222.0.57 () {54 vars in 925 bytes} [Wed Aug 24 16:37:17 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:37:17.038724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dac39-9cea-4301-8c8b-e00f29deee19 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:17.039010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dac39-9cea-4301-8c8b-e00f29deee19 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:17.039291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dac39-9cea-4301-8c8b-e00f29deee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:17.096291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ec6dac39-9cea-4301-8c8b-e00f29deee19 None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:17.097491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ec6dac39-9cea-4301-8c8b-e00f29deee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:17.116199 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:17.172223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 774/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 24 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:21.026171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2930d-af20-47b6-98ab-e1fa24d62c37 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:21.026675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2930d-af20-47b6-98ab-e1fa24d62c37 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:21.026675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2930d-af20-47b6-98ab-e1fa24d62c37 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:21.028010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 745/1519] 10.222.0.52 () {54 vars in 961 bytes} [Wed Aug 24 16:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:21.033510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea795-a48f-46ac-bf34-6f0219acb8bb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:21.033893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea795-a48f-46ac-bf34-6f0219acb8bb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:21.034025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea795-a48f-46ac-bf34-6f0219acb8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:21.055343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fadea795-a48f-46ac-bf34-6f0219acb8bb None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:21.056557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fadea795-a48f-46ac-bf34-6f0219acb8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:21.064839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 775/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Aug 24 16:37:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:37:21.072723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2930d-af20-47b6-98ab-e1fa24d62c37 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:21.085842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2930d-af20-47b6-98ab-e1fa24d62c37 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:21.090650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945af181-badf-43fd-bea2-d3bc44ca64c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:21.091228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945af181-badf-43fd-bea2-d3bc44ca64c3 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:21.091501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945af181-badf-43fd-bea2-d3bc44ca64c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:21.091766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945af181-badf-43fd-bea2-d3bc44ca64c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:21.100621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945af181-badf-43fd-bea2-d3bc44ca64c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:21.102174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945af181-badf-43fd-bea2-d3bc44ca64c3 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:21.111469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 746/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:27.726282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb73bc-f72a-416c-a93b-52f602427c24 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:27.726897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb73bc-f72a-416c-a93b-52f602427c24 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:27.727015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb73bc-f72a-416c-a93b-52f602427c24 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:27.728473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 776/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:37:27.733155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bc89-e7aa-47bd-8520-11742d7043bf None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:27.733561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bc89-e7aa-47bd-8520-11742d7043bf None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:27.733697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bc89-e7aa-47bd-8520-11742d7043bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:27.752975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d256bc89-e7aa-47bd-8520-11742d7043bf None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:27.754159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d256bc89-e7aa-47bd-8520-11742d7043bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:27.761916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 747/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:27.916304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb73bc-f72a-416c-a93b-52f602427c24 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:27.931171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb73bc-f72a-416c-a93b-52f602427c24 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:27.936581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ee9bf-2ab6-48e8-8477-44bc054c9675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:27.937169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ee9bf-2ab6-48e8-8477-44bc054c9675 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:27.937449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ee9bf-2ab6-48e8-8477-44bc054c9675 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:27.937721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ee9bf-2ab6-48e8-8477-44bc054c9675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:27.947342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ee9bf-2ab6-48e8-8477-44bc054c9675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:27.948943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ee9bf-2ab6-48e8-8477-44bc054c9675 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:27.958550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 777/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:28.794850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29886c54-d8c6-4312-9ed8-efe562ac85fb None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:28.795455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29886c54-d8c6-4312-9ed8-efe562ac85fb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:28.795455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29886c54-d8c6-4312-9ed8-efe562ac85fb None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:28.796792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 748/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:28.801662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f294-273e-44a8-a28a-416a9f885da8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:28.802082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f294-273e-44a8-a28a-416a9f885da8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:28.802217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f294-273e-44a8-a28a-416a9f885da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:28.822237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3183f294-273e-44a8-a28a-416a9f885da8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:28.823450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3183f294-273e-44a8-a28a-416a9f885da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:28.831250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 778/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:28.987287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29886c54-d8c6-4312-9ed8-efe562ac85fb None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:29.002273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29886c54-d8c6-4312-9ed8-efe562ac85fb None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:29.007813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b6a28-c70f-43e4-a5f3-93d00e7e5fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:29.008501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b6a28-c70f-43e4-a5f3-93d00e7e5fef service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:29.008776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b6a28-c70f-43e4-a5f3-93d00e7e5fef service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:29.009042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b6a28-c70f-43e4-a5f3-93d00e7e5fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:29.017067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b6a28-c70f-43e4-a5f3-93d00e7e5fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:29.018619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b6a28-c70f-43e4-a5f3-93d00e7e5fef service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:29.028057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 749/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:33.000084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b074e4a-4dee-4b61-9432-c7b7c674f217 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:33.000658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b074e4a-4dee-4b61-9432-c7b7c674f217 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:33.000658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b074e4a-4dee-4b61-9432-c7b7c674f217 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:33.002285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 779/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:33.009857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a83af-bcec-452d-86f7-5d9a44b932f1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:33.010344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a83af-bcec-452d-86f7-5d9a44b932f1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:33.010457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a83af-bcec-452d-86f7-5d9a44b932f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:33.032474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0a0a83af-bcec-452d-86f7-5d9a44b932f1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:33.034261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0a0a83af-bcec-452d-86f7-5d9a44b932f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:33.042139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 750/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:33.326010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b074e4a-4dee-4b61-9432-c7b7c674f217 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:33.327713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b074e4a-4dee-4b61-9432-c7b7c674f217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:33.408279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b074e4a-4dee-4b61-9432-c7b7c674f217 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:33.411798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06167315-df56-4213-a7c9-293e8f5f3096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:33.413290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06167315-df56-4213-a7c9-293e8f5f3096 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:33.413737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06167315-df56-4213-a7c9-293e8f5f3096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:33.414010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06167315-df56-4213-a7c9-293e8f5f3096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:33.422816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06167315-df56-4213-a7c9-293e8f5f3096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:33.424325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06167315-df56-4213-a7c9-293e8f5f3096 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:33.434259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 780/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:35.329152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c65309-d3c1-4757-9087-05a61f48ade3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:35.329669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c65309-d3c1-4757-9087-05a61f48ade3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:35.329669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c65309-d3c1-4757-9087-05a61f48ade3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:35.331007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 751/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:35.337029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfe2c-57ad-458c-93d8-c176d7d23731 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:35.337458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfe2c-57ad-458c-93d8-c176d7d23731 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:35.337589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfe2c-57ad-458c-93d8-c176d7d23731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:35.357022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ed7cfe2c-57ad-458c-93d8-c176d7d23731 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:35.358203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ed7cfe2c-57ad-458c-93d8-c176d7d23731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:35.366069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 781/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:35.680435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c65309-d3c1-4757-9087-05a61f48ade3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:35.685883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=75 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:35.692173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c65309-d3c1-4757-9087-05a61f48ade3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:35.696351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00dafa-02e4-4062-bc6f-fc5119873ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:35.696946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00dafa-02e4-4062-bc6f-fc5119873ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:35.697226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00dafa-02e4-4062-bc6f-fc5119873ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:35.697500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00dafa-02e4-4062-bc6f-fc5119873ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:35.706287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00dafa-02e4-4062-bc6f-fc5119873ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:35.707698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00dafa-02e4-4062-bc6f-fc5119873ea8 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:35.717652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 752/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:39.379784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce04c8-c13e-454d-889f-fc5d49691916 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:39.380361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce04c8-c13e-454d-889f-fc5d49691916 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:39.380361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce04c8-c13e-454d-889f-fc5d49691916 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:39.382714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 782/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:39.388049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b0757-b7cb-41ed-aa44-c6646ad641c7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:39.388500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b0757-b7cb-41ed-aa44-c6646ad641c7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:39.388604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b0757-b7cb-41ed-aa44-c6646ad641c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:39.408784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3b2b0757-b7cb-41ed-aa44-c6646ad641c7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:39.409962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3b2b0757-b7cb-41ed-aa44-c6646ad641c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:39.418877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 753/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:39.975375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce04c8-c13e-454d-889f-fc5d49691916 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:39.977081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c6ce04c8-c13e-454d-889f-fc5d49691916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:40.045467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:40.059771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce04c8-c13e-454d-889f-fc5d49691916 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:40.063402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9272-c7a0-47d0-aa18-c77d610b28cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:40.064038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9272-c7a0-47d0-aa18-c77d610b28cb service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:40.064322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9272-c7a0-47d0-aa18-c77d610b28cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:40.064601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9272-c7a0-47d0-aa18-c77d610b28cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:40.074165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9272-c7a0-47d0-aa18-c77d610b28cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:40.075580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9272-c7a0-47d0-aa18-c77d610b28cb service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:40.084992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 783/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:42.590616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d012-ac53-41a1-89a2-29f4619fc4cf None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:42.591212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d012-ac53-41a1-89a2-29f4619fc4cf None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:42.591212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d012-ac53-41a1-89a2-29f4619fc4cf None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:42.592599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 754/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:42.598007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47edb-f1de-4d6a-b147-18f4f1b873c1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:42.598464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47edb-f1de-4d6a-b147-18f4f1b873c1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:42.598571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47edb-f1de-4d6a-b147-18f4f1b873c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:42.617615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-64f47edb-f1de-4d6a-b147-18f4f1b873c1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:42.618818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-64f47edb-f1de-4d6a-b147-18f4f1b873c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:42.626666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 784/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:43.185137 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d012-ac53-41a1-89a2-29f4619fc4cf None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:43.186810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b337d012-ac53-41a1-89a2-29f4619fc4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:43.199884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d012-ac53-41a1-89a2-29f4619fc4cf None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:43.203350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e2854-bc91-4685-9bf3-99f15a577218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:43.203988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e2854-bc91-4685-9bf3-99f15a577218 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:43.204268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e2854-bc91-4685-9bf3-99f15a577218 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:43.204539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e2854-bc91-4685-9bf3-99f15a577218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:43.213204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e2854-bc91-4685-9bf3-99f15a577218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:43.214604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e2854-bc91-4685-9bf3-99f15a577218 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:43.223682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 755/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:52.612839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=11 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:53.219704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=14 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:11.136388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.137140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.137140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.162288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.164150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.176782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 785/1540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.186191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.186919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.186919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.221122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.222380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.242903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 756/1541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.248432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.262972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcdf9c-9031-4f00-81a9-973da71cdc51 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.276565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.277427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.277912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.278402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.279569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:11.280113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.282623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.301643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 786/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:11.309896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.326694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02a6f-49c6-4222-91d8-ab172ad49d2c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.340511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279710b-b24e-458a-af17-83cbd8a891a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.341143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279710b-b24e-458a-af17-83cbd8a891a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.341422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279710b-b24e-458a-af17-83cbd8a891a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.341751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279710b-b24e-458a-af17-83cbd8a891a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.342329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279710b-b24e-458a-af17-83cbd8a891a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.344383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279710b-b24e-458a-af17-83cbd8a891a2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.402040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 757/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:11.406788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.422623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9058436-883c-4232-b105-03c497975b2b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.488849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71828eb6-3f1d-4f61-bbee-2bf581cc7fe3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.489179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71828eb6-3f1d-4f61-bbee-2bf581cc7fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.489416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71828eb6-3f1d-4f61-bbee-2bf581cc7fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.500317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.500944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.501227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.501501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.502136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.504071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.549187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-71828eb6-3f1d-4f61-bbee-2bf581cc7fe3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.550288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 787/1544] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:11.551508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-71828eb6-3f1d-4f61-bbee-2bf581cc7fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.555198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.572954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ccd04-7649-49c4-bdc7-81f059c92f60 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.582559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 758/1545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:11.585508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.586542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.586835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.587197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.587853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.591005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.601132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5214-f8e1-4a75-af70-6b3b36b6fd56 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.602080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5214-f8e1-4a75-af70-6b3b36b6fd56 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.602717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5214-f8e1-4a75-af70-6b3b36b6fd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.614500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 788/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:11.619826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.644368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c93dc4-b611-47a1-be5b-bde18befcc6c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.653496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1e5c5214-f8e1-4a75-af70-6b3b36b6fd56 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.655439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1e5c5214-f8e1-4a75-af70-6b3b36b6fd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.662227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.663160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.663655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.664233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.664905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.666642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.671489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 759/1547] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.673670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.674269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.674836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.697510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 789/1548] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:39:11.698434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.714190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.716321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.723975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a172ab8-0ee5-4395-9424-166279760a76 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.730991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.734272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.734582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.734868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.735542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:11.735852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.740284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 760/1549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.742423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.744307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fb09f-f9bc-4302-8d87-ee12412a7bea admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.760982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c9a57-7ac5-495c-b5dc-89f6fb7f5bff None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.770418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 790/1550] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:11.773603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb797279-9d4d-4001-bded-c9111468b87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.774293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb797279-9d4d-4001-bded-c9111468b87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.774569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb797279-9d4d-4001-bded-c9111468b87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.774847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb797279-9d4d-4001-bded-c9111468b87c admin admin] PATH_INFO: `/v3/projects/09380cc0f03944468d21fa2571e992d0/users/05e175cab43245d7b35df0adc8564394/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.775864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa553a5-8ef1-4ba3-bf75-31d6f66b37cc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.776178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa553a5-8ef1-4ba3-bf75-31d6f66b37cc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.776462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa553a5-8ef1-4ba3-bf75-31d6f66b37cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.802492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb797279-9d4d-4001-bded-c9111468b87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09380cc0f03944468d21fa2571e992d0, user_id=05e175cab43245d7b35df0adc8564394, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.806458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb797279-9d4d-4001-bded-c9111468b87c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.810197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9aa553a5-8ef1-4ba3-bf75-31d6f66b37cc None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.811524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9aa553a5-8ef1-4ba3-bf75-31d6f66b37cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.824567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 791/1552] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.825972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.826261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.826543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.827466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 761/1552] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:11 2022] PUT /identity/v3/projects/09380cc0f03944468d21fa2571e992d0/users/05e175cab43245d7b35df0adc8564394/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:11.828819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b402ba-1007-416c-adae-610efa9b0c85 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.829111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b402ba-1007-416c-adae-610efa9b0c85 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.829445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b402ba-1007-416c-adae-610efa9b0c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.850890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.852277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.853628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b6b402ba-1007-416c-adae-610efa9b0c85 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.855320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b6b402ba-1007-416c-adae-610efa9b0c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.922841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 792/1553] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.923852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:11.926578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 762/1554] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.927825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d6001-fee4-4c3d-87b7-e39a98f0e070 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.928148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d6001-fee4-4c3d-87b7-e39a98f0e070 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.929088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d6001-fee4-4c3d-87b7-e39a98f0e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.942393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01733054-05f3-4171-9ab0-b96df05a1833 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:11.948579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:11.949244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.949579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.949769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6c5d6001-fee4-4c3d-87b7-e39a98f0e070 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.949900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.950581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:11.950967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6c5d6001-fee4-4c3d-87b7-e39a98f0e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.951661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:11.951661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:39:11.951661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:39:11.953073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca10362-dab4-410d-bb50-ce97643cf740 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:11.958898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 763/1555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:11.960614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2872fa-c58f-444c-b731-c941c57f2c4e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.961066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2872fa-c58f-444c-b731-c941c57f2c4e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.962346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2872fa-c58f-444c-b731-c941c57f2c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:11.984409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1c2872fa-c58f-444c-b731-c941c57f2c4e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:11.985696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1c2872fa-c58f-444c-b731-c941c57f2c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:11.986709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 793/1556] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:11.988240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f06e1-7cab-40b4-8fd6-4bd095dfa580 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:11.988572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f06e1-7cab-40b4-8fd6-4bd095dfa580 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:11.988912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f06e1-7cab-40b4-8fd6-4bd095dfa580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.008044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:12.014196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b38f06e1-7cab-40b4-8fd6-4bd095dfa580 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.016213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b38f06e1-7cab-40b4-8fd6-4bd095dfa580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.054928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 764/1557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.057131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.057408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.057743 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.082508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 794/1558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.083936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe135-70fd-42d8-990a-f803088fb3b7 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.084409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe135-70fd-42d8-990a-f803088fb3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.085385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe135-70fd-42d8-990a-f803088fb3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.092001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.093244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.101558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 765/1559] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.102366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.106705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1cdbe135-70fd-42d8-990a-f803088fb3b7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.107974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1cdbe135-70fd-42d8-990a-f803088fb3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.114300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728744b6-db64-40ab-a4de-281d6508bf53 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.117191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 795/1560] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.119540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.119867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.120236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.121985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.122702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.123044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.123361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.124175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:12.124632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.125870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:12.125870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:39:12.125870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:39:12.126884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9ec6-0442-4acc-b054-c3810487bce8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.128868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:12.141445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.142691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.142911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 766/1561] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:12.144188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.144505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.144977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.155582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 796/1562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.155582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.162885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62092030-b336-42b3-9af8-8255b83ebb8c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.167838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.168467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.168583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.168743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.169019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.169640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.169758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.171638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.177735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 767/1563] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.178593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.179892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.195537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 797/1564] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.196540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.206278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6dc49-c43d-4517-9272-6ef614c291fb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.212332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.212913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.213215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.213496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.214131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:12.214432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.216070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af022be3-79fa-42c6-8d10-60fdc850d8d1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.228010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 798/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:12.229203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33e9c1-308c-4c31-aa76-63f4ae63869b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.229490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33e9c1-308c-4c31-aa76-63f4ae63869b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.229771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33e9c1-308c-4c31-aa76-63f4ae63869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.240661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6853f-3e8c-4c0a-9cb3-a1594dc72bae None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.246291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.246903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.247180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.247448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.249554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ac33e9c1-308c-4c31-aa76-63f4ae63869b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.250927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ac33e9c1-308c-4c31-aa76-63f4ae63869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.257648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.259184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.259932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 799/1566] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.263775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.263775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.263775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.269688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 768/1567] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:12.270772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.282878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0333-4086-4a51-bc4a-ace3d1eccb2d service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.283782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.285422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.288639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.289123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.289471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.289813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.290583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:12.290910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.292602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.295128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 800/1568] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:12.296150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.307592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9c0a6-16c4-4391-a443-95df073e336a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.307739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 769/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:12.308602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.312347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.312959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.313239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.313510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.314771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.316260 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.320942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780b8f-fc02-4d1b-a38f-1d86d27e38e5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.327716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.328463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.328838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.329583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.330387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:12.331046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.332994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.346937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 801/1570] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.348147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.359510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 770/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:12.359510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.373026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062bce79-8bbe-4cf9-ad08-9c56464c93f4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.438376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04efe2-64ce-46fd-8f17-ff0c6db5b3ff admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.438648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.439113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.439392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.439665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.440361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.442401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.446005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.446746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.447063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.447473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.448647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.451087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.463901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 802/1572] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:12.464506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.479500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac89f97-dcb1-4431-8849-e3183f96f2f6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.482057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 771/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.485285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.506176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:12.552197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.552813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.553093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.553984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.554622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:12.554930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.556644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.559337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:12.572902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 803/1574] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:12.573748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.577184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec81455-583a-4cdd-b6e9-38db72f1b21d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.583169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.583807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.584131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.584418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.584659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60020821-dd2b-4ad6-841c-b4e97c338354 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.585393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:12.585731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.587471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.589477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.590288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.590595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.590868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.591496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.593426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.608123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 772/1575] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:12.608123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.622084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b62e-d91d-4e0f-b20f-e353430ce240 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.627174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.627840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.628157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.628431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.629110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.634012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 804/1576] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.634183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:12.634183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:39:12.634183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:39:12.634399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.634489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.662156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 773/1577] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/users => generated 294 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.662156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.673265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3d89c-e78f-4eb4-80c5-a93d0406c20e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.713746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed8d74-9831-488d-8169-e11f76f53e9f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.719841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.720508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.721053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.721373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.722427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.725861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.745093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.745909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.746280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.746565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.747515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.749468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.765905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 774/1578] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:12.765905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.770968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 805/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.771913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.782564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041cbe9-3c59-4ca1-9bf6-f0de3a9c9e82 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.854130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bd22c-573d-4adc-ac94-4263a8253b43 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.857606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.858225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.858507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.858778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.859266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.859547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.860264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.860724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.861168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.861528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.862131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.864247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.892269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 775/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.893380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.904516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 806/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:12.904516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:12.912515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c0a3-4af3-4ab3-bbd3-53c7035c12a2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.913619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:12.975525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:12.976054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8833a1-6505-4b4b-8f87-8d03b8ecacc3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:12.983026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.983597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.983891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.984200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.985039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.987039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:12.989151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:12.989947 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.990352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.990720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.991758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:12.994059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.023313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 807/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/users => generated 284 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:13.024995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.036494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 776/1583] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:13.037325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.039255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736ee8e-5068-4f5d-9dd7-d2dcef49fa50 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.115451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.116789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.117285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.117730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.119232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.121016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.135997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 808/1584] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.135997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.139531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7566e-9155-4535-b00b-1cabc724ee9b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.145511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.146621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.146789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.147198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] PATH_INFO: `/v3/projects/364a00263e804efe83ae759d9c4c01fb/users/fded55a41aae41c6af61382a7c35372e/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.152981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fdeb4-02f1-415c-bf4d-cf65763e0482 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.160766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.161462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.161951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.162305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.163475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.166072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.167088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364a00263e804efe83ae759d9c4c01fb, user_id=fded55a41aae41c6af61382a7c35372e, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.170068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.189848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 777/1585] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:13 2022] PUT /identity/v3/projects/364a00263e804efe83ae759d9c4c01fb/users/fded55a41aae41c6af61382a7c35372e/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:13.190693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.196306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 809/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:13 2022] POST /identity/v3/users => generated 288 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:13.197010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.212693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0746b7-5fc5-41d1-a5ed-62fefa28d701 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.277255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1de2b-c861-46c5-ba6a-446362f59d78 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.284116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.285682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.286153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.286612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.288481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.289483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.290164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.290459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.290732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.291460 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.291657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.293641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.307928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 778/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.309157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.318918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 810/1588] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:13 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:13.319914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.322472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365731d4-37e0-4277-a76c-2252d220ba93 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.327375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:13.330001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.330876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.331227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.331619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.332390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.332867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59637c-40ee-466d-bb28-cbc81d7f655b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.334287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.340867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906196d7-46fe-4f74-82d1-ef203049a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.341860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906196d7-46fe-4f74-82d1-ef203049a37c admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.342500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906196d7-46fe-4f74-82d1-ef203049a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.343045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906196d7-46fe-4f74-82d1-ef203049a37c admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.343877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906196d7-46fe-4f74-82d1-ef203049a37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.347438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906196d7-46fe-4f74-82d1-ef203049a37c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.353105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 779/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.354075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.367055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d91271-16ae-4650-9d88-fc32a7de670f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.374094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.374853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.375152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.375506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.376451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.378314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.390945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 811/1590] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:13 2022] POST /identity/v3/users => generated 287 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:13.390945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.390945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.391314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.394697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 780/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.394697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.406787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaca59e-2083-445b-85fd-8026daca5056 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.412325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.413631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.414061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.414410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.415365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.417319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.420779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:13.422329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:13.432326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 781/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.433117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 812/1593] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:13.433278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.434044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.448489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865e814-8229-4431-a469-bcc55a96dafa admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.452205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6431-7865-48ac-8f7c-27218dd81419 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.457740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.458390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.458680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.459127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.460559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.461695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.461833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.462842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.463209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.463628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.464408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.466623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.474391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 813/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.476597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.487104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 782/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.488057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.493597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649f5b1-6c06-4ec9-8589-40958eed346f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.501081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.501827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab655c-8cdc-4d42-b9f9-aff5a7c2eefd admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.502279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.502804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.503314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.504828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.506746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.507026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.507876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.508189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.508467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.509353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.511058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.516853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:13.523026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 814/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.524398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.528874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 783/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.530086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.538533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063188a-b031-459d-8959-f32abbe33bbb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.542203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e007b-9c90-4f4b-ac02-94c634a4d0d1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.544739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.545710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.546220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.546734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] PATH_INFO: `/v3/projects/cdf93612acce4a868bf2105a6567855b/users/79d593b826c64087bac576d2204302b4/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.547568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.548431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.548714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.549040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.549921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.551905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.564378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 784/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.565297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.574121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf93612acce4a868bf2105a6567855b, user_id=79d593b826c64087bac576d2204302b4, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.576236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:13.576236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:39:13.576236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:39:13.580183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.605571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 815/1599] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:13 2022] PUT /identity/v3/projects/cdf93612acce4a868bf2105a6567855b/users/79d593b826c64087bac576d2204302b4/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:13.606689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.651564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0352c7-2f78-4072-a64c-7bb762ae02b2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.693536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0a2e6-6f09-4322-814e-8838928fb09f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.700722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.701837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.702129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.702413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.703075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.704939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.720957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.721615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.721946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.722257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.723213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 816/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:13.724185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.738863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.739099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d7dd8-ee52-4758-b966-6c82e8e456d3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.739214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.746233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.750861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.751333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.751821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] PATH_INFO: `/v3/projects/02df7e01cd074fc88ecbe4a054500fea/users/252f69423e3045d28a73f952c21f8116/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.753497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 785/1601] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:13.754458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.771084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833c1bd-7d0d-4bae-bf93-77b28b7191f0 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.787318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.790278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.791082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.791082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] PATH_INFO: `/v3/projects/34c80816d21d41b6a3d512df43c1b13a/users/e4117d3363484f89b5b1cb73fb8b4444/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.799223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=02df7e01cd074fc88ecbe4a054500fea, user_id=252f69423e3045d28a73f952c21f8116, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.803303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.826074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c80816d21d41b6a3d512df43c1b13a, user_id=e4117d3363484f89b5b1cb73fb8b4444, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.830390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.849226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 817/1602] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:13 2022] PUT /identity/v3/projects/02df7e01cd074fc88ecbe4a054500fea/users/252f69423e3045d28a73f952c21f8116/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:13.851316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.873894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 786/1603] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:13 2022] PUT /identity/v3/projects/34c80816d21d41b6a3d512df43c1b13a/users/e4117d3363484f89b5b1cb73fb8b4444/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:13.875222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:14.007837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a0a0-8352-4a2c-9790-37064e80afaa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:14.010425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7212b-5e8e-49f4-a37a-f28c52a2a04c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:14.017549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:14.018562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.019613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.020189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] PATH_INFO: `/v3/projects/83bac1da3fae45278144ba6c555629cb/users/1ad05e879f994f0693f2b00e52b6b21f/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.021474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf83152-2964-40cf-9cf9-e1c3b451bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:14.022316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf83152-2964-40cf-9cf9-e1c3b451bbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.022788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf83152-2964-40cf-9cf9-e1c3b451bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.023184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf83152-2964-40cf-9cf9-e1c3b451bbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.023988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf83152-2964-40cf-9cf9-e1c3b451bbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:14.025914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf83152-2964-40cf-9cf9-e1c3b451bbfd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:14.062849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 818/1604] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:14.066639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cae4d-9c20-4be7-be31-cf889c5a6050 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.067211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cae4d-9c20-4be7-be31-cf889c5a6050 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.067335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cae4d-9c20-4be7-be31-cf889c5a6050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.072546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83bac1da3fae45278144ba6c555629cb, user_id=1ad05e879f994f0693f2b00e52b6b21f, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:14.079580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:14.085839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:14.111472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-131cae4d-9c20-4be7-be31-cf889c5a6050 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:14.112777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-131cae4d-9c20-4be7-be31-cf889c5a6050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:14.115055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 787/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:13 2022] PUT /identity/v3/projects/83bac1da3fae45278144ba6c555629cb/users/1ad05e879f994f0693f2b00e52b6b21f/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:14.116702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:14.140184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:14.223051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 819/1606] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:14.224260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e910c-b7b0-4d5f-9dd8-6c182f4f3809 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:14.224784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.225601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.225951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.231091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149ff75-854d-4550-9e47-9f8d655cdfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:14.231881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149ff75-854d-4550-9e47-9f8d655cdfc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.232291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149ff75-854d-4550-9e47-9f8d655cdfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.232671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149ff75-854d-4550-9e47-9f8d655cdfc7 admin admin] PATH_INFO: `/v3/projects/d19a2f819c4744a8b4a89a83f60fbfe5/users/d97889e197354ce7ac3a7ead6d524043/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.251472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149ff75-854d-4550-9e47-9f8d655cdfc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19a2f819c4744a8b4a89a83f60fbfe5, user_id=d97889e197354ce7ac3a7ead6d524043, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:14.254464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149ff75-854d-4550-9e47-9f8d655cdfc7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:14.278924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] MFA Rules not processed for user `252f69423e3045d28a73f952c21f8116`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:14.283347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:14.286606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 788/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:14 2022] PUT /identity/v3/projects/d19a2f819c4744a8b4a89a83f60fbfe5/users/d97889e197354ce7ac3a7ead6d524043/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:14.289065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b085efd-a6f6-4cca-a4a4-ca2c9c7175b6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.289400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b085efd-a6f6-4cca-a4a4-ca2c9c7175b6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.289855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b085efd-a6f6-4cca-a4a4-ca2c9c7175b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.401359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4b085efd-a6f6-4cca-a4a4-ca2c9c7175b6 None None] MFA Rules not processed for user `e4117d3363484f89b5b1cb73fb8b4444`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:14.401359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4b085efd-a6f6-4cca-a4a4-ca2c9c7175b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:14.476269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 820/1608] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:14.478621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:14.574992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 789/1609] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:14.574992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add728b6-cb78-480a-812a-099602b47f2a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.574992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add728b6-cb78-480a-812a-099602b47f2a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.574992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add728b6-cb78-480a-812a-099602b47f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.614208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-add728b6-cb78-480a-812a-099602b47f2a None None] MFA Rules not processed for user `1ad05e879f994f0693f2b00e52b6b21f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:14.616993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-add728b6-cb78-480a-812a-099602b47f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:14.654522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c717-34f8-4b4c-b111-7ea6d53130b0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:14.660339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d12ea-3752-45f3-9505-decccbfe4d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:14.661334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d12ea-3752-45f3-9505-decccbfe4d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.662190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d12ea-3752-45f3-9505-decccbfe4d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.662872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d12ea-3752-45f3-9505-decccbfe4d92 admin admin] PATH_INFO: `/v3/projects/12724c76dc63440d970f4044ac3fadee/users/f6835c202a804a1c8fb6fafd7fb8c307/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.690975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d12ea-3752-45f3-9505-decccbfe4d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12724c76dc63440d970f4044ac3fadee, user_id=f6835c202a804a1c8fb6fafd7fb8c307, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:14.694692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d12ea-3752-45f3-9505-decccbfe4d92 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:14.713270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 790/1610] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:14.714669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add728b6-cb78-480a-812a-099602b47f2a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:14.729589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:14.745327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 821/1611] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:14 2022] PUT /identity/v3/projects/12724c76dc63440d970f4044ac3fadee/users/f6835c202a804a1c8fb6fafd7fb8c307/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:14.749738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.750054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.751993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:14.826068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] MFA Rules not processed for user `d97889e197354ce7ac3a7ead6d524043`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:14.827863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:14.830796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add728b6-cb78-480a-812a-099602b47f2a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:14.833942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:14.939111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 822/1612] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:14.940213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:14.942938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:14.944914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:14.945236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:14.945605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.058580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:15.061252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:15.079597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 791/1613] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:15.081744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:15.089983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be903baf-d6fa-48b2-b387-4bfd54ec03d3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:15.101207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c2eeb-d85c-49cd-9f11-22311eb1c1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:15.102068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c2eeb-d85c-49cd-9f11-22311eb1c1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.102544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c2eeb-d85c-49cd-9f11-22311eb1c1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.103015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c2eeb-d85c-49cd-9f11-22311eb1c1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.103861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c2eeb-d85c-49cd-9f11-22311eb1c1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:15.109043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c2eeb-d85c-49cd-9f11-22311eb1c1d3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:15.171126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca7ee8-823b-412d-8c7a-160011937ed1 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:15.173740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 823/1614] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:14 2022] POST /identity/v3/projects => generated 371 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:15.175528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.176532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.177824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.260464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:15.260711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] MFA Rules not processed for user `f6835c202a804a1c8fb6fafd7fb8c307`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:15.262200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:15.325143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe72f9-1d51-4eab-9e71-f21b5d9638eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:15.325937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe72f9-1d51-4eab-9e71-f21b5d9638eb service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.326957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe72f9-1d51-4eab-9e71-f21b5d9638eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.327245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe72f9-1d51-4eab-9e71-f21b5d9638eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.427603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe72f9-1d51-4eab-9e71-f21b5d9638eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:15.431443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe72f9-1d51-4eab-9e71-f21b5d9638eb service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:15.473982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 792/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:15.477831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377acb4-64b0-4630-8891-25dc1fe6d5a1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.478116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377acb4-64b0-4630-8891-25dc1fe6d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.478391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377acb4-64b0-4630-8891-25dc1fe6d5a1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.482483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 793/1616] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 16:39:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:15.483205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377acb4-64b0-4630-8891-25dc1fe6d5a1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:15.500556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 824/1617] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:15.502960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:15.612730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:15.758938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871f0c2-4046-4676-85a8-6895098e15e8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:15.759139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377acb4-64b0-4630-8891-25dc1fe6d5a1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:15.769098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7e691-8d8a-4644-935e-460af87b2dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:15.769745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e691-8d8a-4644-935e-460af87b2dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.770026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e691-8d8a-4644-935e-460af87b2dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.770302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e691-8d8a-4644-935e-460af87b2dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.774089 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7e691-8d8a-4644-935e-460af87b2dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:15.777792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feede2c9-ba88-44c7-b99f-a3417f3afd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:15.778957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede2c9-ba88-44c7-b99f-a3417f3afd49 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.779189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7e691-8d8a-4644-935e-460af87b2dd0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:15.779342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede2c9-ba88-44c7-b99f-a3417f3afd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.779676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feede2c9-ba88-44c7-b99f-a3417f3afd49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.780580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feede2c9-ba88-44c7-b99f-a3417f3afd49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:15.783271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feede2c9-ba88-44c7-b99f-a3417f3afd49 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:15.826025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 825/1618] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:15 2022] POST /identity/v3/users => generated 292 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:15.830546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ebb5a-5032-49e1-96c3-d1cdc8c45d5e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.830833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ebb5a-5032-49e1-96c3-d1cdc8c45d5e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.832239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ebb5a-5032-49e1-96c3-d1cdc8c45d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.852776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 794/1619] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:15 2022] POST /identity/v3/projects => generated 371 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:15.856501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee9de8-1f8f-422c-8762-a82bbb080386 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.856817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee9de8-1f8f-422c-8762-a82bbb080386 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.857168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee9de8-1f8f-422c-8762-a82bbb080386 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.863319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 795/1620] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 16:39:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:15.864189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee9de8-1f8f-422c-8762-a82bbb080386 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:15.932077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3d6ebb5a-5032-49e1-96c3-d1cdc8c45d5e None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:15.933333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3d6ebb5a-5032-49e1-96c3-d1cdc8c45d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:15.993159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee9de8-1f8f-422c-8762-a82bbb080386 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.008009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.009708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.009955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.010294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.010994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.012748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.029511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 796/1621] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:16.030681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.049392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720a9f-44a7-49d6-90ed-69b197173148 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.057962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.058782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.059070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.059491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.061061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.063867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.075116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:16.093096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 826/1622] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 16:39:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:16.094650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.094943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.095216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.128675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 797/1623] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:16 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:16.133845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.149597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:16.150846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:16.171254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900538b1-904f-4091-9a60-c72684b5bf51 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.175733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 827/1624] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:16.176622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.198203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.199038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.199374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.202455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] PATH_INFO: `/v3/projects/896a918fed814461ad6217fb043f7773/users/436128f89a0849eebaa1a720527d9b0e/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.204280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c105-b371-49b1-b8cf-02744d5818aa None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.213821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.214454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.214762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.215078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.247729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] RBAC: Authorizing `identity:create_grant(project_id=896a918fed814461ad6217fb043f7773, user_id=436128f89a0849eebaa1a720527d9b0e, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.259884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.302094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 798/1625] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:16 2022] PUT /identity/v3/projects/896a918fed814461ad6217fb043f7773/users/436128f89a0849eebaa1a720527d9b0e/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:16.306456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.358487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.362063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.398924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 828/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:16.399932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.427466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832371-52f3-436b-932e-8daa0c29a033 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.482706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:16.491612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757c28d-3069-443f-a2ae-2188d28c660c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.514994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.515630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.515918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.516211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.516858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.518469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.544656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 799/1627] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:16.548028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.580748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63793ce-3c06-4e9a-9e5d-9e2891263839 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.625545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.626292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.626671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.627049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.644270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.645165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.645600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.646052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.646907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.649054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.699435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 800/1628] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:16.700580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.726011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f84386-371e-4dd7-9955-74bf51c34939 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.752670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.753677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.754115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.754561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] PATH_INFO: `/v3/projects/da42885dae514fc295c15d69b27716e5/users/850589b8ef964ab48a54788be4ada90d/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.800631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da42885dae514fc295c15d69b27716e5, user_id=850589b8ef964ab48a54788be4ada90d, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.803790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.820094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.822000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.853474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 829/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:16.854501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.880885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 801/1630] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:16 2022] PUT /identity/v3/projects/da42885dae514fc295c15d69b27716e5/users/850589b8ef964ab48a54788be4ada90d/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:16.885791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.886248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33141df-2a96-453b-a243-eb17dff41509 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:17.052775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc1595-d3f1-4e1e-bb27-20cb46399e9c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:17.060329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:17.061051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.061333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.061608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.062267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:17.063964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:17.091211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 802/1631] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:17.092335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:17.114244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee33bb-2697-480e-a514-e051be33e4c1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:17.128175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1032eb0-6069-472c-afc3-a88193ff12a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:17.128861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1032eb0-6069-472c-afc3-a88193ff12a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.129142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1032eb0-6069-472c-afc3-a88193ff12a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.129422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1032eb0-6069-472c-afc3-a88193ff12a6 admin admin] PATH_INFO: `/v3/projects/da42885dae514fc295c15d69b27716e5/users/850589b8ef964ab48a54788be4ada90d/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.130059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619701a4-2e49-4cef-bd75-07cbfe6f6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:17.133084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619701a4-2e49-4cef-bd75-07cbfe6f6ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.136826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619701a4-2e49-4cef-bd75-07cbfe6f6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.137222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1032eb0-6069-472c-afc3-a88193ff12a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da42885dae514fc295c15d69b27716e5, user_id=850589b8ef964ab48a54788be4ada90d, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:17.137355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619701a4-2e49-4cef-bd75-07cbfe6f6ea5 admin admin] PATH_INFO: `/v3/projects/896a918fed814461ad6217fb043f7773/users/436128f89a0849eebaa1a720527d9b0e/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.140279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1032eb0-6069-472c-afc3-a88193ff12a6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:17.143477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619701a4-2e49-4cef-bd75-07cbfe6f6ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896a918fed814461ad6217fb043f7773, user_id=436128f89a0849eebaa1a720527d9b0e, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:17.153713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619701a4-2e49-4cef-bd75-07cbfe6f6ea5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:17.173740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 803/1632] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:17 2022] PUT /identity/v3/projects/da42885dae514fc295c15d69b27716e5/users/850589b8ef964ab48a54788be4ada90d/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:17.173740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b94f8c-1635-47b7-832a-c05c3742e8d2 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.173740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b94f8c-1635-47b7-832a-c05c3742e8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.173740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b94f8c-1635-47b7-832a-c05c3742e8d2 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.173740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 804/1633] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 16:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:17.176472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.176472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.176472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.204982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 830/1634] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:16 2022] PUT /identity/v3/projects/896a918fed814461ad6217fb043f7773/users/436128f89a0849eebaa1a720527d9b0e/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:17.204982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3f53-8cdb-4c1d-937e-a671abcfabc2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.204982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3f53-8cdb-4c1d-937e-a671abcfabc2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.204982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3f53-8cdb-4c1d-937e-a671abcfabc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.240126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:17.248812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f7ab3f53-8cdb-4c1d-937e-a671abcfabc2 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:17.251714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f7ab3f53-8cdb-4c1d-937e-a671abcfabc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:17.270885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] MFA Rules not processed for user `850589b8ef964ab48a54788be4ada90d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:17.277475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:17.383508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:17.463599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 831/1635] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:17.464958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be11f23-5e08-4347-8dd1-7822033716f9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.465530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be11f23-5e08-4347-8dd1-7822033716f9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.465912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be11f23-5e08-4347-8dd1-7822033716f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.509892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 805/1636] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:17.509892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:17.525578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5be11f23-5e08-4347-8dd1-7822033716f9 None None] MFA Rules not processed for user `436128f89a0849eebaa1a720527d9b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:17.538519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5be11f23-5e08-4347-8dd1-7822033716f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:17.661593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d58243-289b-4d2c-8a41-8da49838de40 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:17.670588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:17.671489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.671784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.672145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.705888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:17.707479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:17.746654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 806/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:17.748053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:17.767129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 832/1638] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:17.769915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b66dae-c13d-4a0b-9f19-0e7afe6b0e86 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:17.770483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648abdd-ade7-449b-8fe5-b95aaa6b36f0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.770847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648abdd-ade7-449b-8fe5-b95aaa6b36f0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.771217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648abdd-ade7-449b-8fe5-b95aaa6b36f0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.773480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 833/1639] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 16:39:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:17.780568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.780936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.781298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.787538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:17.788734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.789198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.789830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.821802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:17.825586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:17.856281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:17.856896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 807/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:17.858557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:17.863384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:17.880784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3e104-e4f8-4336-ac6e-3282b2ef0d95 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:17.895583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:17.896672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:17.897278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:17.897818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:17.917190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 834/1641] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:17.934967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:17.936833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:17.962339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 808/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:18.012189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:18.031619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b6ac6-e209-4716-876a-0fef352c3cb0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:18.046184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:18.047139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:18.049921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:18.050426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:18.080209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:18.083045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:18.143761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 835/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:18.471630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:18.535729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:18.552442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad757a2-b359-4430-8d14-15f9e1385d8e service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:18.563284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f956e70-6ca0-4726-8d4c-058eb545fd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:18.564163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f956e70-6ca0-4726-8d4c-058eb545fd3d service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:18.571166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f956e70-6ca0-4726-8d4c-058eb545fd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:18.574238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f956e70-6ca0-4726-8d4c-058eb545fd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:18.695664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44a0ba-fb94-44c9-a12a-679f99a9ae5b service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:18.713687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:18.714347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:18.714627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:18.714914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] PATH_INFO: `/v3/users/1ad05e879f994f0693f2b00e52b6b21f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:18.716684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad05e879f994f0693f2b00e52b6b21f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:18.720345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:18.746766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:18.758329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f956e70-6ca0-4726-8d4c-058eb545fd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:18.759033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f956e70-6ca0-4726-8d4c-058eb545fd3d service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:18.776960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 836/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:18.848779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad05e879f994f0693f2b00e52b6b21f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:18.876398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad05e879f994f0693f2b00e52b6b21f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:18.906452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad05e879f994f0693f2b00e52b6b21f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:18.909069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad05e879f994f0693f2b00e52b6b21f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:18.977362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad05e879f994f0693f2b00e52b6b21f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:19.013616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 809/1645] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:18 2022] DELETE /identity/v3/users/1ad05e879f994f0693f2b00e52b6b21f => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:19.023150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:19.023536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:19.024329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:19.108841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:19.125510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:19.219580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:19.264068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 837/1646] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:19.279324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:19.279971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:19.475490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8357-15fa-40d6-806a-82d41e052248 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:19.486146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123c79a-95db-49ed-bf1f-90638d08e92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:19.486814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123c79a-95db-49ed-bf1f-90638d08e92b service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:19.487091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123c79a-95db-49ed-bf1f-90638d08e92b service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:19.487380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123c79a-95db-49ed-bf1f-90638d08e92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:19.521498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123c79a-95db-49ed-bf1f-90638d08e92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:19.524054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123c79a-95db-49ed-bf1f-90638d08e92b service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:19.592687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 810/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:20.532870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:20.534459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:20.576565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05669f4-3ee2-4d59-9596-3edca252d077 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:20.601667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:20.605889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:20.606275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:20.606928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] PATH_INFO: `/v3/users/d97889e197354ce7ac3a7ead6d524043` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:20.611523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d97889e197354ce7ac3a7ead6d524043)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:20.613123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:20.613123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Aug 24 16:39:20.613123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: Aug 24 16:39:20.620354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:20.887420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd97889e197354ce7ac3a7ead6d524043'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:20.922655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd97889e197354ce7ac3a7ead6d524043'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:20.925447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd97889e197354ce7ac3a7ead6d524043'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:20.950540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd97889e197354ce7ac3a7ead6d524043'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.003172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd97889e197354ce7ac3a7ead6d524043'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.030565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:21.031551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:21.032079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:21.047346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 838/1648] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:20 2022] DELETE /identity/v3/users/d97889e197354ce7ac3a7ead6d524043 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:21.091126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:21.094538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:21.219206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:21.220957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:21.415097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 811/1649] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:21.472130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:21.473802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:21.500577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcefcf-5f49-4d40-9a4f-3b4d146a846b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:21.523863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:21.547073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:21.548725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:21.549170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] PATH_INFO: `/v3/projects/83bac1da3fae45278144ba6c555629cb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:21.550749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=83bac1da3fae45278144ba6c555629cb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:21.554541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:21.630081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:21.750364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41dd6-d9a3-48a1-b97f-f9e350000534 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:21.832146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83bac1da3fae45278144ba6c555629cb', 'request_id': 'req-085a2b33-e890-4289-adcf-3b31c456f4ae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.878951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83bac1da3fae45278144ba6c555629cb', 'request_id': 'req-085a2b33-e890-4289-adcf-3b31c456f4ae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.916468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83bac1da3fae45278144ba6c555629cb', 'request_id': 'req-085a2b33-e890-4289-adcf-3b31c456f4ae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.934120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:21.965197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] The token cache is being invalidate because project 83bac1da3fae45278144ba6c555629cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:21.965599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.968387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-085a2b33-e890-4289-adcf-3b31c456f4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:21.976202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 839/1650] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:21 2022] DELETE /identity/v3/projects/83bac1da3fae45278144ba6c555629cb => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:21.978069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b20bd-47e5-4470-b2b5-baf3e60114a5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:21.984899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b20bd-47e5-4470-b2b5-baf3e60114a5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:21.985501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b20bd-47e5-4470-b2b5-baf3e60114a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.017508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:22.018155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.019512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.020155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.037518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-202b20bd-47e5-4470-b2b5-baf3e60114a5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:22.038971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:22.039508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-202b20bd-47e5-4470-b2b5-baf3e60114a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:22.214833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:22.216417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:22.244493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 840/1651] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:22.247253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 812/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:22.253075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.253411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.254001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.322608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:22.323861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:22.354557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 841/1653] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:22.359160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:22.371836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a543b-55d8-4a54-9ec6-e072697f56e6 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:22.399163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:22.400287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.400716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.401153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.401990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:22.402447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:22.404295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:22.442952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 813/1654] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:22.449130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:22.459823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feeb3e5-a54a-436d-b097-3981b963d37f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:22.489922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:22.492482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.498996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.500171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.501388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:22.503531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:22.564102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:22.567023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:22.590927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 842/1655] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:22 2022] POST /identity/v3/projects => generated 363 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:22.592515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f812-4aac-4655-9099-63a86c43bd2c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:22.593112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:22.690947 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:22.691828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.692144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.692422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] PATH_INFO: `/v3/users/f6835c202a804a1c8fb6fafd7fb8c307` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.694222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6835c202a804a1c8fb6fafd7fb8c307)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:22.697045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:22.747150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:22.762052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d709b-72bb-444b-9588-4288f03d502a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:22.774652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:22.775664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.776206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.778198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.779104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:22.783128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6835c202a804a1c8fb6fafd7fb8c307'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:22.784221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:22.799593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6835c202a804a1c8fb6fafd7fb8c307'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:22.815111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 843/1656] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:22 2022] POST /identity/v3/users => generated 282 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:22.816823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6835c202a804a1c8fb6fafd7fb8c307'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:22.817510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:22.818501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6835c202a804a1c8fb6fafd7fb8c307'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:22.833865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4f43cecd-3b4f-4c41-8ddf-6158d34d24ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6835c202a804a1c8fb6fafd7fb8c307'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:22.850734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 814/1657] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:22 2022] DELETE /identity/v3/users/f6835c202a804a1c8fb6fafd7fb8c307 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:22.853693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae480-c370-45ef-b1f4-e92b43238116 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.854148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae480-c370-45ef-b1f4-e92b43238116 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.854571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ae480-c370-45ef-b1f4-e92b43238116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:22.859620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b645e9-611c-4987-803f-1a874c0dde96 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:22.896365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-932ae480-c370-45ef-b1f4-e92b43238116 None None] MFA Rules not processed for user `05e175cab43245d7b35df0adc8564394`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:22.898289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-932ae480-c370-45ef-b1f4-e92b43238116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:22.988042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 815/1658] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:22.988042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:22.988042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:22.988042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.028307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:23.029519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:23.033189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:23.046310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e31a61-b1ec-4121-9773-7b4e10fccfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:23.047429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31a61-b1ec-4121-9773-7b4e10fccfcc admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.047928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31a61-b1ec-4121-9773-7b4e10fccfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.048425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31a61-b1ec-4121-9773-7b4e10fccfcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.050083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e31a61-b1ec-4121-9773-7b4e10fccfcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:23.052443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e31a61-b1ec-4121-9773-7b4e10fccfcc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:23.070540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 844/1659] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:23.072922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.073529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.073916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.106910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] MFA Rules not processed for user `05e175cab43245d7b35df0adc8564394`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:23.107138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:23.133291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 845/1660] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:23.134428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:23.137801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:23.167939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae949-d117-4601-a0f3-75a62e485f6e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:23.177666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:23.181176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.181544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.181989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.184095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:23.188409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:23.228957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 846/1661] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:23.233727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:23.238283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 816/1662] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:23.239820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:23.242711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:23.264257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bc56-b284-4d5a-a2f0-057ff583251b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:23.281204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:23.281905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.284155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.284451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] PATH_INFO: `/v3/projects/1a314db606534878b7ae671a2a346d0d/users/7ed99d8af40b443aaaad7bab82ee7b82/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.342564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a314db606534878b7ae671a2a346d0d, user_id=7ed99d8af40b443aaaad7bab82ee7b82, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:23.345374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:23.377799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 817/1663] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:23 2022] PUT /identity/v3/projects/1a314db606534878b7ae671a2a346d0d/users/7ed99d8af40b443aaaad7bab82ee7b82/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:23.381812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:23.386934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:23.422902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6894dd-1d76-413c-9770-9bc756b033cb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:23.437020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:23.531395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cebd6-3932-453d-b333-a19ee12def18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:23.532272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cebd6-3932-453d-b333-a19ee12def18 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.532690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cebd6-3932-453d-b333-a19ee12def18 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.533233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cebd6-3932-453d-b333-a19ee12def18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.566181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456905b-9065-4104-a74c-b2d008caee62 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:23.571295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:23.572001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.572804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.572804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.655919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cebd6-3932-453d-b333-a19ee12def18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:23.659917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cebd6-3932-453d-b333-a19ee12def18 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:23.689017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 847/1664] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:23.704522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.704522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.705942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.712458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:23.716557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:23.745095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 818/1665] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:23.778085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:23.797059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] MFA Rules not processed for user `7ed99d8af40b443aaaad7bab82ee7b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:23.798637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:23.893272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 848/1666] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:23.893952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:23.910583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed9014-de2c-4bb9-b440-e39d04f8c064 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:23.935350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:23.937076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:23.938147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:23.940402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:23.942164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:23.951233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:24.005358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:24.007636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:24.032568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 819/1667] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:23 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:24.057805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:24.067279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c1e3-ec95-4272-bf6d-18eaa9337c6b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:24.331807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:24.332514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:24.332803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:24.333155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] PATH_INFO: `/v3/projects/d19a2f819c4744a8b4a89a83f60fbfe5` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:24.345036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19a2f819c4744a8b4a89a83f60fbfe5)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:24.347536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:24.349239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ac727-6060-41e1-a558-c518c5c868fc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:24.366984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:24.367722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:24.368104 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:24.368626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:24.370267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:24.377178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:24.443150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 820/1668] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:24 2022] POST /identity/v3/users => generated 288 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:24.447970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:24.476329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19a2f819c4744a8b4a89a83f60fbfe5', 'request_id': 'req-7d462067-16de-47d0-8921-cb50e9f89573'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.477546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557bfb5-fdb7-46d0-bb35-c9876b396d27 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:24.491988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19a2f819c4744a8b4a89a83f60fbfe5', 'request_id': 'req-7d462067-16de-47d0-8921-cb50e9f89573'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.517115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19a2f819c4744a8b4a89a83f60fbfe5', 'request_id': 'req-7d462067-16de-47d0-8921-cb50e9f89573'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.538729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] The token cache is being invalidate because project d19a2f819c4744a8b4a89a83f60fbfe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:24.539360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.541667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.545534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 849/1669] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:24 2022] DELETE /identity/v3/projects/d19a2f819c4744a8b4a89a83f60fbfe5 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:24.549796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:24.555236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:24.572689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:24.637558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:24.702701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:24.703572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:24.703985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:24.704375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:24.705144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:24.706937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:24.733108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 821/1670] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:24.737005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:24.738330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d462067-16de-47d0-8921-cb50e9f89573 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:24.739581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:24.754839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:24.755509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:24.755786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:24.757156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] PATH_INFO: `/v3/users/850589b8ef964ab48a54788be4ada90d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:24.760743 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=850589b8ef964ab48a54788be4ada90d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:24.763776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:24.786870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d7d00-62f6-437f-ab4a-d84ded57828a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:24.807989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:24.813954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:24.813954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:24.813954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] PATH_INFO: `/v3/projects/9240a6e5c4ac4cbb8c4eabde1471126c/users/8c12060fb261443fbe060d964ff1fc09/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:24.850455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9240a6e5c4ac4cbb8c4eabde1471126c, user_id=8c12060fb261443fbe060d964ff1fc09, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:24.854388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:24.896324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850589b8ef964ab48a54788be4ada90d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.902853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 822/1671] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:24 2022] PUT /identity/v3/projects/9240a6e5c4ac4cbb8c4eabde1471126c/users/8c12060fb261443fbe060d964ff1fc09/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:24.909043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:24.923916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850589b8ef964ab48a54788be4ada90d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.924942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850589b8ef964ab48a54788be4ada90d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.945016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850589b8ef964ab48a54788be4ada90d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.961411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2eee9775-9ad8-4393-8832-9d683f01bbc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850589b8ef964ab48a54788be4ada90d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:24.976727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 850/1672] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:24 2022] DELETE /identity/v3/users/850589b8ef964ab48a54788be4ada90d => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:24.984336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c50d7-552f-4d69-a991-b7237a43f61c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:24.984925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c50d7-552f-4d69-a991-b7237a43f61c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:24.986196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c50d7-552f-4d69-a991-b7237a43f61c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.001712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 851/1673] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 16:39:24 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:25.009052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5fd62-4e76-49d9-9e55-19c9d9cca9bb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.009489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5fd62-4e76-49d9-9e55-19c9d9cca9bb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.010768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5fd62-4e76-49d9-9e55-19c9d9cca9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.081430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5118-1ad2-447a-a154-c0bb8e58b254 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:25.112076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f8a5fd62-4e76-49d9-9e55-19c9d9cca9bb None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:25.116731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f8a5fd62-4e76-49d9-9e55-19c9d9cca9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.154247 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:25.156139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.156797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.157408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.158626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:25.161688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:25.178528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 823/1674] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:25.180792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:25.183012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.238182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90f325-0c16-4cc1-ac62-55b2349eef07 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:25.253326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:25.254268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.254678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.255062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] PATH_INFO: `/v3/projects/9240a6e5c4ac4cbb8c4eabde1471126c/users/8c12060fb261443fbe060d964ff1fc09/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.260315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9240a6e5c4ac4cbb8c4eabde1471126c, user_id=8c12060fb261443fbe060d964ff1fc09, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:25.266690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:25.276294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 852/1675] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:25.302445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 824/1676] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:25 2022] PUT /identity/v3/projects/9240a6e5c4ac4cbb8c4eabde1471126c/users/8c12060fb261443fbe060d964ff1fc09/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:25.304799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.305142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.305523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.342689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] MFA Rules not processed for user `8c12060fb261443fbe060d964ff1fc09`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:25.344257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.350557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:25.356546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.464537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:25.465991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:25.519357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234f606-b639-4b54-ae0c-de7c11bdf993 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:25.533057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:25.534646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.534945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.535219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.554376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:25.558488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:25.574089 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 853/1677] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:25.575041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:25.578654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.585709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 825/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:25.587385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:25.591592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.720898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b0af0-f36d-4168-9c16-07468aaa3834 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:25.728056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:25.728721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.729003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.729278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] PATH_INFO: `/v3/projects/12724c76dc63440d970f4044ac3fadee` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.730559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12724c76dc63440d970f4044ac3fadee)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:25.732522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:25.747814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e398-b6f9-45f2-bcc3-c980488c2470 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:25.758453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf420e4-1f2a-48f5-82b8-a85f4676f52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:25.759191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf420e4-1f2a-48f5-82b8-a85f4676f52d service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.759507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf420e4-1f2a-48f5-82b8-a85f4676f52d service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.759749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf420e4-1f2a-48f5-82b8-a85f4676f52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.787681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf420e4-1f2a-48f5-82b8-a85f4676f52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:25.789070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf420e4-1f2a-48f5-82b8-a85f4676f52d service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:25.812941 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 826/1679] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:25.836584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12724c76dc63440d970f4044ac3fadee', 'request_id': 'req-f0e8c990-0f55-44d5-9bfb-122551eae098'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:25.850854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12724c76dc63440d970f4044ac3fadee', 'request_id': 'req-f0e8c990-0f55-44d5-9bfb-122551eae098'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:25.866705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12724c76dc63440d970f4044ac3fadee', 'request_id': 'req-f0e8c990-0f55-44d5-9bfb-122551eae098'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:25.887085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] The token cache is being invalidate because project 12724c76dc63440d970f4044ac3fadee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:25.887397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:25.888298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:25.891093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 854/1680] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:25 2022] DELETE /identity/v3/projects/12724c76dc63440d970f4044ac3fadee => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:25.896400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89e93-5578-459f-93d1-b3cec30f200d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:25.896999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89e93-5578-459f-93d1-b3cec30f200d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:25.897809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af89e93-5578-459f-93d1-b3cec30f200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:25.945146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4af89e93-5578-459f-93d1-b3cec30f200d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:25.947470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4af89e93-5578-459f-93d1-b3cec30f200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:25.972539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:25.976185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.046181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 827/1681] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:26.054564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.055101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.055674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.109268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:26.112562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.136267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 828/1682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:26.137556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:26.141763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c990-0f55-44d5-9bfb-122551eae098 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:26.142091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.156145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:26.156973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.157471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.157820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.161825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.175739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:26.182250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3177c85-3b7d-49d8-90d5-fb6e72507356 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:26.199388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:26.200618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.200618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.200618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] PATH_INFO: `/v3/projects/da42885dae514fc295c15d69b27716e5` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.201983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da42885dae514fc295c15d69b27716e5)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:26.203053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:26.203053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Aug 24 16:39:26.203053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: Aug 24 16:39:26.204415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:26.276187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:26.277954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:26.321077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 855/1683] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:26.324909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:26.348139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72342-a53b-47a8-82a6-5cc3e736a68c service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:26.394432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da42885dae514fc295c15d69b27716e5', 'request_id': 'req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:26.427829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da42885dae514fc295c15d69b27716e5', 'request_id': 'req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:26.440401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da42885dae514fc295c15d69b27716e5', 'request_id': 'req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:26.443491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:26.470584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] The token cache is being invalidate because project da42885dae514fc295c15d69b27716e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:26.471223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:26.473035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65b77de5-f3fb-4a0b-aa40-1d7bc6f6589c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:26.476596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 829/1684] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:26 2022] DELETE /identity/v3/projects/da42885dae514fc295c15d69b27716e5 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:26.481495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d843f5-e8c3-4cc7-a5e6-f30d041e13d1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.482308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d843f5-e8c3-4cc7-a5e6-f30d041e13d1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.482831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d843f5-e8c3-4cc7-a5e6-f30d041e13d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.535310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-04d843f5-e8c3-4cc7-a5e6-f30d041e13d1 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:26.540519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-04d843f5-e8c3-4cc7-a5e6-f30d041e13d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.599576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:26.600847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.601138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.601732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.602952 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:26.603577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:26.605247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:26.648825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 856/1685] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:26.653026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:26.657972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.688906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd4167-7793-4416-8246-1728aeb4cf6a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:26.696578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:26.697215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.697497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.697780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.698425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:26.706298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:26.727761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 830/1686] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:26.740751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.741915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.742484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.786316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 857/1687] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:26 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:26.787403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:26.791421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:26.793192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:26.932017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 831/1688] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:26.935000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:26.956654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c466-5629-44d6-82eb-13976eb03737 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:26.960397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882bdc6-c7b3-4696-9fd1-10ffffa5073e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:26.972942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:26.975869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.977087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.977512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.978456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9325d41-426e-4cae-8d8b-4664e4ddc1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:26.979799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:26.980232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:26.980640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9325d41-426e-4cae-8d8b-4664e4ddc1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:26.981880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9325d41-426e-4cae-8d8b-4664e4ddc1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:26.982257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9325d41-426e-4cae-8d8b-4664e4ddc1e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:26.983554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d268535-5885-4595-90b2-99f847768fd4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:26.984398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9325d41-426e-4cae-8d8b-4664e4ddc1e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:26.989531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9325d41-426e-4cae-8d8b-4664e4ddc1e7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.055818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 832/1689] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:39:27.059358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26007a4d-25fc-47d1-8817-aca56938e612 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.061412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26007a4d-25fc-47d1-8817-aca56938e612 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.062412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26007a4d-25fc-47d1-8817-aca56938e612 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.067308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 833/1690] 10.222.0.52 () {54 vars in 961 bytes} [Wed Aug 24 16:39:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:27.068368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26007a4d-25fc-47d1-8817-aca56938e612 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.088538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 858/1691] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:26 2022] POST /identity/v3/users => generated 303 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:27.091428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.096533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.097598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.119895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26007a4d-25fc-47d1-8817-aca56938e612 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.135135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246f4ec-6313-4b81-9db9-51280b0fc2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.138427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246f4ec-6313-4b81-9db9-51280b0fc2df admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.141062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246f4ec-6313-4b81-9db9-51280b0fc2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.141467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246f4ec-6313-4b81-9db9-51280b0fc2df admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.143292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246f4ec-6313-4b81-9db9-51280b0fc2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.151298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246f4ec-6313-4b81-9db9-51280b0fc2df admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.169785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:27.195932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] MFA Rules not processed for user `79d593b826c64087bac576d2204302b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:27.197319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:27.218969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 834/1692] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:27 2022] POST /identity/v3/projects => generated 357 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:27.221153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.221649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.222138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.263754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:27.265233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:27.364283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 859/1693] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:27.365951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.400389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd9d04-1a99-4f8b-8a87-290447019c2e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.414892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:27.442050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 835/1694] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Aug 24 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:27.443603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.544747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562533e-df0d-4111-9d78-3cd7d68d810b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.546637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.547440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.547781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.549038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.549770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.551854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.552231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.552595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.554062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.554376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.555831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.558456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.575535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 860/1695] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:27.576738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.579120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:27.593674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 836/1696] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:27 2022] POST /identity/v3/users => generated 279 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:27.595032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.610368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355bc2a-eb56-4e58-8b20-62270f2e2dbf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.620398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.621360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.621840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.622290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.624617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:27.641649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.643264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.659250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 837/1697] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:27.663512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.682702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec72a1-500c-4b8b-9f32-e51299a73c39 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.689722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.690704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.691297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.691766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.692814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.696203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.716031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 838/1698] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:27.717093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.731386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795e65d-6e80-4c5c-99f3-c3132253d8b3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.733993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179cb8d-723e-4de2-906e-e1726ca91c4e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.737277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.738050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.738501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.738915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.739730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.741776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aede101-7b4a-436f-92ad-a76b83b038dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.742408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aede101-7b4a-436f-92ad-a76b83b038dc service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.742779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aede101-7b4a-436f-92ad-a76b83b038dc service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.743066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aede101-7b4a-436f-92ad-a76b83b038dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.743311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.759892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 839/1699] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:27.760901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.773319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc833a8-1a0d-43be-a448-7d777c1b584f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:27.779675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:27.780898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.781196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.781472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] PATH_INFO: `/v3/projects/eec494dc03b14ce98d328eef462b786c/users/a3b769d4c0d0408cb6e007cece8d7405/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.806853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec494dc03b14ce98d328eef462b786c, user_id=a3b769d4c0d0408cb6e007cece8d7405, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.811764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.839935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 840/1700] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:27 2022] PUT /identity/v3/projects/eec494dc03b14ce98d328eef462b786c/users/a3b769d4c0d0408cb6e007cece8d7405/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:27.841541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:27.873678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:27.914693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aede101-7b4a-436f-92ad-a76b83b038dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:27.917025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aede101-7b4a-436f-92ad-a76b83b038dc service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:27.930406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 861/1701] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:27.932286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:27.932728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:27.933145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:27.970271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] MFA Rules not processed for user `a3b769d4c0d0408cb6e007cece8d7405`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:27.985026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:28.025655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248553c-18ef-459b-997d-f9b186854686 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.044102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.045332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.045332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.045778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.046569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.048565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.066907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 841/1702] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:28.071911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.101435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c56663-203b-427c-92a1-c7b5a6b5d4e0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.120038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef23b9-005a-4799-aba3-86bb0a3b479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.120959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef23b9-005a-4799-aba3-86bb0a3b479e admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.121399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef23b9-005a-4799-aba3-86bb0a3b479e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.121903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef23b9-005a-4799-aba3-86bb0a3b479e admin admin] PATH_INFO: `/v3/projects/d304aba7e3aa44ab9f9fdc1a5ea4049e/users/ec2024714f744b8fabc1a2cc4dd4c4fd/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.166121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 862/1703] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:28.167071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.180352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef23b9-005a-4799-aba3-86bb0a3b479e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d304aba7e3aa44ab9f9fdc1a5ea4049e, user_id=ec2024714f744b8fabc1a2cc4dd4c4fd, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.195682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef23b9-005a-4799-aba3-86bb0a3b479e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.200178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0406a-ab4e-43b3-a0ce-f34fc7fa2ff2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.218788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.219469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.219764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.220075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.220502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:28.220737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.222765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.241987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 842/1704] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:28 2022] PUT /identity/v3/projects/d304aba7e3aa44ab9f9fdc1a5ea4049e/users/ec2024714f744b8fabc1a2cc4dd4c4fd/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:28.260741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.263721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.265399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.329561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 863/1705] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:28 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:28.330454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.410968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] MFA Rules not processed for user `ec2024714f744b8fabc1a2cc4dd4c4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:28.414461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:28.501110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a8a4d-0048-461a-99f1-acac605d489a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.510462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.512840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.512840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.513032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.513538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.515494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.588390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 864/1706] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:28 2022] POST /identity/v3/users => generated 309 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:28.590069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.611603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473a8e9-ba3d-45e2-ae7e-c2771db7dc7a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.626306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.627001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.627395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.627559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.631746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.633375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.688860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 865/1707] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:28.689887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.715141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab391db-5c8b-44ad-93dc-3315779da2e1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.736669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 843/1708] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:28.741408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.744782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.745476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.745764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.746045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] PATH_INFO: `/v3/projects/e74d1b237dbb4032871324b9d5ca5154/users/bbb107961fb74d31bfef359874c9c7c1/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.779457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7f0d9-6232-406a-9611-2cb4d910aa42 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:28.793858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74d1b237dbb4032871324b9d5ca5154, user_id=bbb107961fb74d31bfef359874c9c7c1, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.797185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.818888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:28.818888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:28.818888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:28.818888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:28.819744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:28.826140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:28.827535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:28.854358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 866/1709] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:28 2022] PUT /identity/v3/projects/e74d1b237dbb4032871324b9d5ca5154/users/bbb107961fb74d31bfef359874c9c7c1/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:28.859100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:28.954839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 844/1710] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:28 2022] POST /identity/v3/projects => generated 355 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:28.958686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:29.101674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c40e8-e966-4ab1-a71c-4ba4050ab6c9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:29.220025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72347c-936d-4870-83d4-94db8ee024a1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:29.250868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:29.253061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.255194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.257033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.258078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:29.263888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:29.302844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:29.304549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.305069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.305521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.306404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:29.309564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:29.339219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 845/1711] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:28 2022] POST /identity/v3/users => generated 284 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:29.340440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:29.351696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 867/1712] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:29.353079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:29.380808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ab57c-ed65-4cc4-816f-2d867b78b1e1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:29.386751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2530e10-de11-4d6b-a456-7424be290b31 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:29.392498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dda62-8e0a-4c44-83b8-421cb9b84f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:29.393774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dda62-8e0a-4c44-83b8-421cb9b84f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.394115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dda62-8e0a-4c44-83b8-421cb9b84f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.394404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dda62-8e0a-4c44-83b8-421cb9b84f99 admin admin] PATH_INFO: `/v3/projects/e74d1b237dbb4032871324b9d5ca5154/users/bbb107961fb74d31bfef359874c9c7c1/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.396886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dda62-8e0a-4c44-83b8-421cb9b84f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74d1b237dbb4032871324b9d5ca5154, user_id=bbb107961fb74d31bfef359874c9c7c1, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:29.397971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:29.398659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.398938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.399218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.399878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:29.400422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dda62-8e0a-4c44-83b8-421cb9b84f99 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:29.407066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:29.424976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 868/1713] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:29 2022] PUT /identity/v3/projects/e74d1b237dbb4032871324b9d5ca5154/users/bbb107961fb74d31bfef359874c9c7c1/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:29.441641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99140c00-9be4-4d69-882b-ba47c6879c85 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.441960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99140c00-9be4-4d69-882b-ba47c6879c85 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.442317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99140c00-9be4-4d69-882b-ba47c6879c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.458485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 846/1714] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:29.462724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:29.507695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-99140c00-9be4-4d69-882b-ba47c6879c85 None None] MFA Rules not processed for user `bbb107961fb74d31bfef359874c9c7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:29.508889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9c7fd-73ab-403f-8300-61eb5e1e5f09 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:29.509214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-99140c00-9be4-4d69-882b-ba47c6879c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:29.676616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:29.729377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:29.730294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.731019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.731433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] PATH_INFO: `/v3/projects/9c6f7627090142678484fbb32b42a834/users/43a69e9ada4641d7869ae7cc11fdb1c7/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.761830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 869/1715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:29.766155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.766507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.768386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:29.781074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6f7627090142678484fbb32b42a834, user_id=43a69e9ada4641d7869ae7cc11fdb1c7, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:29.786580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:29.815978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:29.825732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 847/1716] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:29 2022] PUT /identity/v3/projects/9c6f7627090142678484fbb32b42a834/users/43a69e9ada4641d7869ae7cc11fdb1c7/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:29.825732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:29.830073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:29.844804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] MFA Rules not processed for user `fded55a41aae41c6af61382a7c35372e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:29.848266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:29.975553 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d16914-dd69-49ab-b944-3507fed9200d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:29.985873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff0c01-6b41-4dee-9cf2-c921435d581c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:29.988671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff0c01-6b41-4dee-9cf2-c921435d581c service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:29.989123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff0c01-6b41-4dee-9cf2-c921435d581c service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:29.990047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff0c01-6b41-4dee-9cf2-c921435d581c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:30.020127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 870/1717] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:30.021089 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:30.203171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79bf15-6cb2-4f99-8ebd-16ac82d16c12 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:30.204699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff0c01-6b41-4dee-9cf2-c921435d581c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:30.206182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff0c01-6b41-4dee-9cf2-c921435d581c service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:30.219047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:30.220541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:30.220541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:30.220541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:30.221303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:30.227728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:30.230029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 848/1718] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:29 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:30.233187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669356fa-3d21-41b1-8800-31df3497c834 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:30.233481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669356fa-3d21-41b1-8800-31df3497c834 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:30.236070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669356fa-3d21-41b1-8800-31df3497c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:30.249142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 871/1719] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:30.263388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:30.295705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb6378-cd69-4725-9c2a-f32130dd0141 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:30.310711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:30.311936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-669356fa-3d21-41b1-8800-31df3497c834 None None] MFA Rules not processed for user `fded55a41aae41c6af61382a7c35372e`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:30.314175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:30.314462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:30.314744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] PATH_INFO: `/v3/projects/9c6f7627090142678484fbb32b42a834/users/43a69e9ada4641d7869ae7cc11fdb1c7/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:30.318521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-669356fa-3d21-41b1-8800-31df3497c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:30.319366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6f7627090142678484fbb32b42a834, user_id=43a69e9ada4641d7869ae7cc11fdb1c7, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:30.323346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:30.368141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 849/1720] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:30.379439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 872/1721] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:30 2022] PUT /identity/v3/projects/9c6f7627090142678484fbb32b42a834/users/43a69e9ada4641d7869ae7cc11fdb1c7/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:30.391185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:30.391702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:30.392164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:30.392571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:30.420670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:30.455663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] MFA Rules not processed for user `43a69e9ada4641d7869ae7cc11fdb1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:30.457150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:30.626632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18822a3d-9b56-4661-a4e3-5314ae3ea452 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:30.668865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:30.669644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:30.669837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:30.670059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:30.725922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:30.734075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:30.776586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 850/1722] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:30.783231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:30.785914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:30.885502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:30.887291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:30.907173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 873/1723] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:31.064766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6dd79-d02e-477a-9f15-d0180e3664e9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:31.074644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:31.075322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:31.075583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:31.075979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] PATH_INFO: `/v3/users/e4117d3363484f89b5b1cb73fb8b4444` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:31.087774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4117d3363484f89b5b1cb73fb8b4444)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:31.089118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:31.091266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:31.112671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73266da0-cc40-4c19-8a79-47a615f77e4f service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:31.148788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:31.151938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:31.152913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:31.153343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] PATH_INFO: `/v3/users/a3b769d4c0d0408cb6e007cece8d7405` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:31.166403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b769d4c0d0408cb6e007cece8d7405)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:31.171163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:31.216775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4117d3363484f89b5b1cb73fb8b4444'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.244309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4117d3363484f89b5b1cb73fb8b4444'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.268340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4117d3363484f89b5b1cb73fb8b4444'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.270218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4117d3363484f89b5b1cb73fb8b4444'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.301586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4117d3363484f89b5b1cb73fb8b4444'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.316923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b769d4c0d0408cb6e007cece8d7405'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.321284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 851/1724] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:30 2022] DELETE /identity/v3/users/e4117d3363484f89b5b1cb73fb8b4444 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:31.322991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:31.325994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:31.348220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b769d4c0d0408cb6e007cece8d7405'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.354847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b769d4c0d0408cb6e007cece8d7405'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.384024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b769d4c0d0408cb6e007cece8d7405'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.414190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-74884e10-a81c-44bf-b39c-2ae066b2efaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b769d4c0d0408cb6e007cece8d7405'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:31.435170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 874/1725] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:31 2022] DELETE /identity/v3/users/a3b769d4c0d0408cb6e007cece8d7405 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:31.444737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc4874-e2ae-4361-939f-5d7445fadecd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:31.444737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc4874-e2ae-4361-939f-5d7445fadecd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:31.444967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc4874-e2ae-4361-939f-5d7445fadecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:31.487744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7ecc4874-e2ae-4361-939f-5d7445fadecd None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:31.494222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ecc4874-e2ae-4361-939f-5d7445fadecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:31.494751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a75b3-afc6-43a1-8bfc-cc660ab007a5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:31.574988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:31.633168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:31.633812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:31.634094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:31.637050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:31.640790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:31.654696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 875/1726] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:31.656421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173acca-318c-4745-8005-10be65c51951 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:31.656719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173acca-318c-4745-8005-10be65c51951 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:31.656999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173acca-318c-4745-8005-10be65c51951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:31.719741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1173acca-318c-4745-8005-10be65c51951 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:31.725476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1173acca-318c-4745-8005-10be65c51951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:31.754230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 876/1727] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:31.755349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173acca-318c-4745-8005-10be65c51951 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:31.760103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1173acca-318c-4745-8005-10be65c51951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:31.867711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:31.869556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:31.870778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:31.907682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 852/1728] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:31.909409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:31.915929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:31.962198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173acca-318c-4745-8005-10be65c51951 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:31.976905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:31.978175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:31.978778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:31.981117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.045038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:32.048160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:32.070309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 877/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:32.072203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84738f-a283-4810-8afa-1ad3c89051e9 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:32.081446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1ad79-35fd-43df-ab7f-efc13571113e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:32.082342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ad79-35fd-43df-ab7f-efc13571113e service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:32.082739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ad79-35fd-43df-ab7f-efc13571113e service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:32.083025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1ad79-35fd-43df-ab7f-efc13571113e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.119032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1ad79-35fd-43df-ab7f-efc13571113e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:32.119935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1ad79-35fd-43df-ab7f-efc13571113e service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:32.158084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 853/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:32.175642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:32.178836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:32.200223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed731a-d834-4b1a-9e50-29be8147f19d service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:32.213043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:32.213861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:32.214327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:32.214825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.216918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:32.310564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:32.312158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a94c20-5212-49f4-a204-59e8999d4d14 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:32.322159 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 878/1731] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:32.434424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b6bb-20d8-493c-ac27-a3aa4af6ebcc None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:32.435324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b6bb-20d8-493c-ac27-a3aa4af6ebcc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:32.435798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b6bb-20d8-493c-ac27-a3aa4af6ebcc None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.438261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 854/1732] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 16:39:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:32.462516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec01f-c4e4-496b-9d74-9c7247291054 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:32.462746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec01f-c4e4-496b-9d74-9c7247291054 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:32.462974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec01f-c4e4-496b-9d74-9c7247291054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.517903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2aeec01f-c4e4-496b-9d74-9c7247291054 None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:32.520839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2aeec01f-c4e4-496b-9d74-9c7247291054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:32.532043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 879/1733] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:39:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:32.572576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b6bb-20d8-493c-ac27-a3aa4af6ebcc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:32.585598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b6bb-20d8-493c-ac27-a3aa4af6ebcc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:32.589431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:32.590355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:32.590901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:32.591173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.604840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:32.606291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:32.622094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 855/1734] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:32.723847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:32.724245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:32.725625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:32.746321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] MFA Rules not processed for user `79d593b826c64087bac576d2204302b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:32.748455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:32.831671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 880/1735] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:33.179330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:33.180919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:33.227043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e4bf2-e880-4201-b4d4-2020f833ece6 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:33.247236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:33.247903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.248216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.248490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] PATH_INFO: `/v3/projects/34c80816d21d41b6a3d512df43c1b13a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.249740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c80816d21d41b6a3d512df43c1b13a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:33.251628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:33.344950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:33.346442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:33.374011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85546b-9517-46ee-a688-a5cac7716967 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:33.410143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c80816d21d41b6a3d512df43c1b13a', 'request_id': 'req-39be87f4-d9ba-415a-8246-104bd8a06535'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.434787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c80816d21d41b6a3d512df43c1b13a', 'request_id': 'req-39be87f4-d9ba-415a-8246-104bd8a06535'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.462261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c80816d21d41b6a3d512df43c1b13a', 'request_id': 'req-39be87f4-d9ba-415a-8246-104bd8a06535'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.469617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:33.495438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] The token cache is being invalidate because project 34c80816d21d41b6a3d512df43c1b13a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:33.496163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.497668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39be87f4-d9ba-415a-8246-104bd8a06535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.499478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 856/1736] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:33 2022] DELETE /identity/v3/projects/34c80816d21d41b6a3d512df43c1b13a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:33.512568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b71de6-d8b9-475b-bdf3-19d605a9d136 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.513180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b71de6-d8b9-475b-bdf3-19d605a9d136 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.513566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b71de6-d8b9-475b-bdf3-19d605a9d136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.533734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:33.534648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.534938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.535215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] PATH_INFO: `/v3/projects/eec494dc03b14ce98d328eef462b786c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.537753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec494dc03b14ce98d328eef462b786c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:33.542599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:33.563974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a0b71de6-d8b9-475b-bdf3-19d605a9d136 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:33.565725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a0b71de6-d8b9-475b-bdf3-19d605a9d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:33.576707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:33.588622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 857/1737] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:33.597622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.598172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.598421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.639619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:33.642085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:33.679432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 858/1738] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:33.681267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:33.693670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fce3d6-7b86-4787-aed5-bbd66687f641 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:33.722872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec494dc03b14ce98d328eef462b786c', 'request_id': 'req-9482c6bc-8095-4393-b31b-ae9d09559d5a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.754600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec494dc03b14ce98d328eef462b786c', 'request_id': 'req-9482c6bc-8095-4393-b31b-ae9d09559d5a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.781943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec494dc03b14ce98d328eef462b786c', 'request_id': 'req-9482c6bc-8095-4393-b31b-ae9d09559d5a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.814701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] The token cache is being invalidate because project eec494dc03b14ce98d328eef462b786c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:33.815018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.816122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:33.822311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 881/1739] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:33 2022] DELETE /identity/v3/projects/eec494dc03b14ce98d328eef462b786c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:33.833639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:33.835087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:33.837552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.840141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.840450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.841692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:33.842295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:33.842604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:33.846700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:33.875520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482c6bc-8095-4393-b31b-ae9d09559d5a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:33.888738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:33.889773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.890223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.903613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] PATH_INFO: `/v3/users/bbb107961fb74d31bfef359874c9c7c1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.906643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb107961fb74d31bfef359874c9c7c1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:33.909155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:33.914749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 859/1740] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:33.918045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:33.920756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:33.938374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761b79a-e13e-496f-bdd2-5ab9f474e2d8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:33.952507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:33.953349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:33.953349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:33.953610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:33.954370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:33.956524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:34.001044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb107961fb74d31bfef359874c9c7c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:34.014144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb107961fb74d31bfef359874c9c7c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:34.015328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb107961fb74d31bfef359874c9c7c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:34.023897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 860/1741] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:33 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:34.029706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:34.038285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb107961fb74d31bfef359874c9c7c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:34.067265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2dad8331-366c-4ce7-9b28-b27586ef440b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb107961fb74d31bfef359874c9c7c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:34.076845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:34.078334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 882/1742] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:33 2022] DELETE /identity/v3/users/bbb107961fb74d31bfef359874c9c7c1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:34.118541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.119049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.119443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.179823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] MFA Rules not processed for user `fded55a41aae41c6af61382a7c35372e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:34.185091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:34.253600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1dd9b-e5e6-464e-b1e5-67af16c6b722 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:34.391103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:34.395871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 883/1743] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:34.421590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.421890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.422170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.425143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:34.437004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:34.457919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:34.493043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 861/1744] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:34 2022] POST /identity/v3/users => generated 302 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:34.494363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:34.511381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb5202-2859-41e9-b71f-0a1b471e0310 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:34.528520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:34.529148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.529427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.529699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.530609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:34.539410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:34.571736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 884/1745] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:34.575986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:34.593711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37151a-ca28-4ad7-b566-86d872bfb73d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:34.688871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c70f4c-2c2c-4706-8ec0-e331cda2123f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:34.690368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c70f4c-2c2c-4706-8ec0-e331cda2123f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.701381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c70f4c-2c2c-4706-8ec0-e331cda2123f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.709567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c70f4c-2c2c-4706-8ec0-e331cda2123f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.713765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c70f4c-2c2c-4706-8ec0-e331cda2123f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:34.730272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c70f4c-2c2c-4706-8ec0-e331cda2123f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:34.773538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 862/1746] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:34.775016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:34.783323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:34.810442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e7d23-84f5-40ff-aa6b-f4a0b1a85d34 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:34.824701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9c1c8-a40f-4478-b286-aafffb9feea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:34.828462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c1c8-a40f-4478-b286-aafffb9feea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.828911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c1c8-a40f-4478-b286-aafffb9feea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.831846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9c1c8-a40f-4478-b286-aafffb9feea0 admin admin] PATH_INFO: `/v3/projects/5b417c71b7714807895bcf1d10fbc1c3/users/a7024ec76baa404db8ce1041302a60bb/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.874552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.874854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.875136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.880653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9c1c8-a40f-4478-b286-aafffb9feea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b417c71b7714807895bcf1d10fbc1c3, user_id=a7024ec76baa404db8ce1041302a60bb, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:34.883477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9c1c8-a40f-4478-b286-aafffb9feea0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:34.911416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 885/1747] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:34 2022] PUT /identity/v3/projects/5b417c71b7714807895bcf1d10fbc1c3/users/a7024ec76baa404db8ce1041302a60bb/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:34.914049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:34.916792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:34.917227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:34.964705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:34.966421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:35.111176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 886/1748] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:35.148316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:35.150048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:35.236071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:35.237972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:35.310543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 863/1749] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:39:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:35.559539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75117814-0dff-4bcb-a370-ce349a8db2f6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:35.592406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:35.593190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:35.596246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:35.596550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:35.628829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:35.632545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:35.648122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 887/1750] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:35.672804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:35.674771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:35.734061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:35.735520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:35.758370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337579bc-ad52-4763-bbc4-abfa81683b6e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:35.800094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:35.801974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:35.802139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:35.803375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] PATH_INFO: `/v3/projects/e74d1b237dbb4032871324b9d5ca5154` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:35.809811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74d1b237dbb4032871324b9d5ca5154)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:35.817384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:36.093217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74d1b237dbb4032871324b9d5ca5154', 'request_id': 'req-f4808101-cd94-44a1-a6ac-97c5668aff3d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:36.117126 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:36.138576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5392-59e2-4adf-8718-93dc302dd99b service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:36.150091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74d1b237dbb4032871324b9d5ca5154', 'request_id': 'req-f4808101-cd94-44a1-a6ac-97c5668aff3d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:36.174969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74d1b237dbb4032871324b9d5ca5154', 'request_id': 'req-f4808101-cd94-44a1-a6ac-97c5668aff3d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:36.213316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] The token cache is being invalidate because project e74d1b237dbb4032871324b9d5ca5154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:36.213654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:36.217617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4808101-cd94-44a1-a6ac-97c5668aff3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:36.225837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 864/1751] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:35 2022] DELETE /identity/v3/projects/e74d1b237dbb4032871324b9d5ca5154 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:36.231611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90ab52-6c53-42bc-960f-1953d77fb0ad None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:36.232282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90ab52-6c53-42bc-960f-1953d77fb0ad None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:36.232783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90ab52-6c53-42bc-960f-1953d77fb0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:36.317637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3f90ab52-6c53-42bc-960f-1953d77fb0ad None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:36.324487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3f90ab52-6c53-42bc-960f-1953d77fb0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:36.368545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:36.415174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 865/1752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:36.423755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:36.423755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:36.423755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:36.423755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:36.424209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63058680-91cf-4840-b299-b126e264288b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:36.424209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63058680-91cf-4840-b299-b126e264288b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:36.424209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63058680-91cf-4840-b299-b126e264288b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:36.428706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:36.492405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-63058680-91cf-4840-b299-b126e264288b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:36.497850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-63058680-91cf-4840-b299-b126e264288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:36.518867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 866/1753] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:36.523102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63058680-91cf-4840-b299-b126e264288b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:36.568813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63058680-91cf-4840-b299-b126e264288b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:36.582229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:36.583018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:36.583483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:36.586007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:36.587598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:36.588429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:36.592817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:36.653354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 867/1754] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:36.657880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:36.668420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:36.670042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:36.676557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d21b1-a711-4a5f-8d31-9fd71106fb07 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:36.692496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:36.693611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:36.693843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:36.694074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:36.694668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:36.696795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:36.700309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 888/1755] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:39:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 967 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:39:36.845831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:36.848279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 868/1756] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:36 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:36.863388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056c301-fc85-46b6-8e2b-1eb10744ee15 service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:37.245728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:37.252155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:37.252973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:37.255063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:37.263608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:37.273197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:37.350684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 889/1757] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:36 2022] POST /identity/v3/users => generated 292 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:37.353534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:37.374994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd228-a440-4762-8f41-d760e8abbbb8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:37.420492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:37.429112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:37.429462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:37.429749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:37.430527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:37.432172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:37.517765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 869/1758] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:37.519142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:37.539903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a33a9-161a-43b2-b6e2-3f35afb59f56 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:37.580471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5861e-2265-4881-964f-988709b5d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:37.581181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5861e-2265-4881-964f-988709b5d6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:37.581462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5861e-2265-4881-964f-988709b5d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:37.581739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5861e-2265-4881-964f-988709b5d6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:37.582420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5861e-2265-4881-964f-988709b5d6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:37.605932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5861e-2265-4881-964f-988709b5d6a3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:37.660563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:37.686501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 890/1759] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:37.701269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72d97e-a464-4d30-9826-decf806ef358 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:37.717146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:37.723001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:37.723403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:37.723756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] PATH_INFO: `/v3/projects/b1d7f1ef6383450f800ed392fd997d50/users/e0f38d1e780c49eda2f94a01459ee6ed/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:37.768601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d7f1ef6383450f800ed392fd997d50, user_id=e0f38d1e780c49eda2f94a01459ee6ed, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:37.782934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:37.826739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 870/1760] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:37 2022] PUT /identity/v3/projects/b1d7f1ef6383450f800ed392fd997d50/users/e0f38d1e780c49eda2f94a01459ee6ed/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:37.843011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aadd53-b960-4e90-a0ce-b0aa2c5b92d0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:37.843344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aadd53-b960-4e90-a0ce-b0aa2c5b92d0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:37.843633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aadd53-b960-4e90-a0ce-b0aa2c5b92d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:37.953724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-59aadd53-b960-4e90-a0ce-b0aa2c5b92d0 None None] MFA Rules not processed for user `e0f38d1e780c49eda2f94a01459ee6ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:37.955313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-59aadd53-b960-4e90-a0ce-b0aa2c5b92d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:38.017027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:38.366353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 891/1761] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 524 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:38.367962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:38.606716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:38.607060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:38.607294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:38.710710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:38.716195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:38.723983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c2e7c-c86e-4baa-ac19-b1a65661b53c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:38.745823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:38.746275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:38.746550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:38.747019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:38.747737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:38.756506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:38.875286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 871/1762] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:38 2022] POST /identity/v3/projects => generated 383 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:38.876608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:38.923638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 892/1763] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 24 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:39.063795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:39.117807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e9f3-bcaf-49b4-b7ea-a6cc2b29944a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:39.140269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:39.140655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.140928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.141255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.149258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:39.150763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:39.175469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:39.177300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:39.205364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ab3a7-0ccc-4633-a4e5-07fee98a48f9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:39.227735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 872/1764] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:38 2022] POST /identity/v3/users => generated 298 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:39.234301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:39.271153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42425591-45fc-4c64-bce7-ef4ef23b60f8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:39.291032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:39.292035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.292035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.292259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.292891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:39.296560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:39.323931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 873/1765] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:39.325908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:39.356474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e49c11-3889-4f2a-ae09-a28c36ec20dd admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:39.382762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:39.383447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.383727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.384038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] PATH_INFO: `/v3/projects/be706cd0f660452d8725caec41617787/users/87fc6422c571402099f24adfd96949ef/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.415594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be706cd0f660452d8725caec41617787, user_id=87fc6422c571402099f24adfd96949ef, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:39.424789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:39.455461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 874/1766] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:39 2022] PUT /identity/v3/projects/be706cd0f660452d8725caec41617787/users/87fc6422c571402099f24adfd96949ef/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:39.460342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:39.495792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:39.496679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.497141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.497548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.508139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:39.510032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:39.541978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 893/1767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:39.630095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8377a-1e35-4579-b6e1-fc324b370dc9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:39.648442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cc326-e4ec-46fd-9171-28e67136d9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:39.649020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc326-e4ec-46fd-9171-28e67136d9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.649297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc326-e4ec-46fd-9171-28e67136d9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.649628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cc326-e4ec-46fd-9171-28e67136d9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.650283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cc326-e4ec-46fd-9171-28e67136d9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:39.651901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cc326-e4ec-46fd-9171-28e67136d9d5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:39.681566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 875/1768] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:39:39.684993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:39.713510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfb8a1-8cb9-418f-bb5c-7aea57cadde8 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:39.736714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:39.741887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.744907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.745501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] PATH_INFO: `/v3/projects/be706cd0f660452d8725caec41617787/users/87fc6422c571402099f24adfd96949ef/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.753455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be706cd0f660452d8725caec41617787, user_id=87fc6422c571402099f24adfd96949ef, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:39.780954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:39.818757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 894/1769] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:39 2022] PUT /identity/v3/projects/be706cd0f660452d8725caec41617787/users/87fc6422c571402099f24adfd96949ef/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:39.828110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99f65-eaf6-44a0-b0b7-23822aa73e26 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:39.828943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99f65-eaf6-44a0-b0b7-23822aa73e26 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:39.829495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a99f65-eaf6-44a0-b0b7-23822aa73e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:39.931179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-08a99f65-eaf6-44a0-b0b7-23822aa73e26 None None] MFA Rules not processed for user `87fc6422c571402099f24adfd96949ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:39.931179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-08a99f65-eaf6-44a0-b0b7-23822aa73e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:40.246841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 876/1770] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:40.249830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:40.252064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:40.473603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5e1b9-62c7-48df-95bd-72c11717fd63 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:40.497274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:40.497908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:40.498432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:40.498717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:40.508977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:40.514196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:40.514196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:40.529273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 895/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:40.585338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:40.585827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:40.586176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:40.713046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] MFA Rules not processed for user `a7024ec76baa404db8ce1041302a60bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:40.714422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:40.732246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:40.737941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:40.750139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c76cb-44fc-406b-8abb-e7c5e7bc1019 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:40.754838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:40.756692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:40.757457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:40.757948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:40.867985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 877/1772] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:40 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:40.871510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:40.875777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:40.878248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:40.881940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:40.921557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 896/1773] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:40.991148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ff49-0861-4b51-8101-8e8ee29ef11e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:41.000254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:41.000887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:41.003396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:41.003677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:41.010684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:41.013304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:41.089441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 878/1774] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:40 2022] POST /identity/v3/projects => generated 405 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:41.092390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:41.108760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5641f-4534-412d-b8cd-c8ea15b895f8 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:41.303578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:41.304249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:41.304771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:41.305057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:41.305707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:41.309066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:41.360692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 897/1775] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:41 2022] POST /identity/v3/users => generated 309 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:41.371831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:41.385758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d8116-fe00-4dff-9d1c-0cced8befedc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:41.391978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:41.392952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:41.393621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:41.393983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:41.394798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:41.397798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:41.435169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 879/1776] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:41.436913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:41.462899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dfff0-b0eb-457a-8b8f-37457824f08e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:41.480513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:41.482154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:41.482687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:41.483187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] PATH_INFO: `/v3/projects/b7109d96dd874fa0b4c0399a1ed58519/users/674166ddd0214e2abf1daaf2920453fc/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:41.520161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7109d96dd874fa0b4c0399a1ed58519, user_id=674166ddd0214e2abf1daaf2920453fc, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:41.524934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:41.546397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 898/1777] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:41 2022] PUT /identity/v3/projects/b7109d96dd874fa0b4c0399a1ed58519/users/674166ddd0214e2abf1daaf2920453fc/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:41.551979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:41.579351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d4f7-3f3b-490d-ac7a-610a55f01c3c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:41.671227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:41.671974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:41.672263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:41.672534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:41.674442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:41.676077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:41.695860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 880/1778] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:41.696174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:41.712256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b233a-3f4f-477c-a980-0275d4a71bb5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:41.743651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6ad5d-ab2f-4142-b192-3246e4fafa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:41.744295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6ad5d-ab2f-4142-b192-3246e4fafa26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:41.744618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6ad5d-ab2f-4142-b192-3246e4fafa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:41.744888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6ad5d-ab2f-4142-b192-3246e4fafa26 admin admin] PATH_INFO: `/v3/projects/b7109d96dd874fa0b4c0399a1ed58519/users/674166ddd0214e2abf1daaf2920453fc/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:41.751310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6ad5d-ab2f-4142-b192-3246e4fafa26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7109d96dd874fa0b4c0399a1ed58519, user_id=674166ddd0214e2abf1daaf2920453fc, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:41.754170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6ad5d-ab2f-4142-b192-3246e4fafa26 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:41.822577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 899/1779] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:41 2022] PUT /identity/v3/projects/b7109d96dd874fa0b4c0399a1ed58519/users/674166ddd0214e2abf1daaf2920453fc/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:42.075903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:42.078047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:42.094127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa9ffb-5803-4969-86b8-981de17615fe admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:42.113338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:42.193627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:42.194553 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:42.195604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:42.196756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] PATH_INFO: `/v3/users/7ed99d8af40b443aaaad7bab82ee7b82` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:42.200035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed99d8af40b443aaaad7bab82ee7b82)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:42.205509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:42.342161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed99d8af40b443aaaad7bab82ee7b82'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:42.355388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed99d8af40b443aaaad7bab82ee7b82'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:42.378114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed99d8af40b443aaaad7bab82ee7b82'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:42.378995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed99d8af40b443aaaad7bab82ee7b82'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:42.400791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed99d8af40b443aaaad7bab82ee7b82'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:42.413889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 881/1780] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:42 2022] DELETE /identity/v3/users/7ed99d8af40b443aaaad7bab82ee7b82 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:42.425887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:42.426680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:42.427137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:42.497883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:42.500214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:42.642724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:42.737471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 900/1781] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:42.803221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:42.806391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:43.025458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46f60-6b62-49bb-8f6c-4a41236c3e96 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:43.085280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c9ea5-334a-4d79-8aeb-116a8bd864d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:43.087340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c9ea5-334a-4d79-8aeb-116a8bd864d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:43.087629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c9ea5-334a-4d79-8aeb-116a8bd864d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:43.087982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c9ea5-334a-4d79-8aeb-116a8bd864d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:43.132600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c9ea5-334a-4d79-8aeb-116a8bd864d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:43.141665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c9ea5-334a-4d79-8aeb-116a8bd864d2 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:43.175199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 882/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:45.069781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:45.072282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:45.093090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd6752-2bf3-4c67-b180-d670efbbd422 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:45.110048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:45.111116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:45.111664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:45.112137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] PATH_INFO: `/v3/users/e0f38d1e780c49eda2f94a01459ee6ed` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:45.113726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f38d1e780c49eda2f94a01459ee6ed)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:45.116990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:45.262053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f38d1e780c49eda2f94a01459ee6ed'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.280220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f38d1e780c49eda2f94a01459ee6ed'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.285567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f38d1e780c49eda2f94a01459ee6ed'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.315926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f38d1e780c49eda2f94a01459ee6ed'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.396814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f38d1e780c49eda2f94a01459ee6ed'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.420142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 901/1783] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:45 2022] DELETE /identity/v3/users/e0f38d1e780c49eda2f94a01459ee6ed => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:45.424872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:45.425403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:45.425633 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:45.476565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:45.479071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:45.567203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:45.648263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 883/1784] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:45.665817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:45.668186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:45.695043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0e5d2-8a05-4218-921d-080958eeec18 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:45.710861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:45.711925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:45.712513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:45.713471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] PATH_INFO: `/v3/projects/1a314db606534878b7ae671a2a346d0d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:45.717862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a314db606534878b7ae671a2a346d0d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:45.720246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:45.722520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:45.729481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:45.831847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a314db606534878b7ae671a2a346d0d', 'request_id': 'req-136b6ef1-f6a7-446e-bb22-79b2842618af'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.842803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a314db606534878b7ae671a2a346d0d', 'request_id': 'req-136b6ef1-f6a7-446e-bb22-79b2842618af'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.854871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a314db606534878b7ae671a2a346d0d', 'request_id': 'req-136b6ef1-f6a7-446e-bb22-79b2842618af'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.877221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] The token cache is being invalidate because project 1a314db606534878b7ae671a2a346d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:45.877800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.878795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:45.883989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 902/1785] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:45 2022] DELETE /identity/v3/projects/1a314db606534878b7ae671a2a346d0d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:45.884150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012fb51-8d32-4172-ad8b-f00a8b5854fd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:45.886645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:45.888915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:46.023974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:46.080922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b6ef1-f6a7-446e-bb22-79b2842618af admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:46.102266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:46.103230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.103626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.104098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] PATH_INFO: `/v3/users/8c12060fb261443fbe060d964ff1fc09` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.104369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:46.105133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.105780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c12060fb261443fbe060d964ff1fc09)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:46.107638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.108118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.108480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:46.118376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:46.163201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:46.164974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402759a1-182e-4611-8664-05ce9e5ee0dc service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:46.254482 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 884/1786] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:46.262309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f4cc0-ff56-455e-b632-d2b14be8185d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.264042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f4cc0-ff56-455e-b632-d2b14be8185d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.264972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f4cc0-ff56-455e-b632-d2b14be8185d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.270505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 885/1787] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:39:46 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:46.277410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c12060fb261443fbe060d964ff1fc09'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:46.289661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c12060fb261443fbe060d964ff1fc09'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:46.290546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c12060fb261443fbe060d964ff1fc09'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:46.310065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c12060fb261443fbe060d964ff1fc09'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:46.333360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76321cb-1870-4516-b44e-60d12ac6758b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c12060fb261443fbe060d964ff1fc09'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:46.351237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 903/1788] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:45 2022] DELETE /identity/v3/users/8c12060fb261443fbe060d964ff1fc09 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:46.419501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.419995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.420379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.482572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:46.486681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15ee9-2b19-4f44-8fa1-ae7d206eca8f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.486963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15ee9-2b19-4f44-8fa1-ae7d206eca8f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.487241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15ee9-2b19-4f44-8fa1-ae7d206eca8f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.487829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:46.488639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 904/1789] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:39:46.512124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.512361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.512653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.581836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:46.583176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:46.750203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 886/1790] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:39:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:46.784911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:46.787782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 905/1791] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:39:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:46.789054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:46.812429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a27ca-03f7-48e9-a043-4aca948e24a5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:46.822111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdde7-675e-4eb5-be06-f7a42fc623f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:46.823304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdde7-675e-4eb5-be06-f7a42fc623f9 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.823968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdde7-675e-4eb5-be06-f7a42fc623f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.826419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdde7-675e-4eb5-be06-f7a42fc623f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.845288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bdde7-675e-4eb5-be06-f7a42fc623f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:46.848682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bdde7-675e-4eb5-be06-f7a42fc623f9 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:46.887323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:46.891391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:46.916562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 887/1792] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:46.936615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379285a3-54ee-482e-a66d-16eceaeab665 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:46.946240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:46.946951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:46.947366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:46.947802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:46.962538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:46.964872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:46.983784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 906/1793] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:49.129593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:49.131291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:49.131983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:49.189195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] MFA Rules not processed for user `674166ddd0214e2abf1daaf2920453fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:49.190641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:49.212528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:49.338953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:49.340657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:49.435023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 888/1794] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:49.504700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:49.549927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d0de4-a343-4d98-b1bb-c206eefcfe15 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:49.555833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:49.556607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:49.557000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:49.557405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] PATH_INFO: `/v3/projects/b1d7f1ef6383450f800ed392fd997d50` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:49.558859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d7f1ef6383450f800ed392fd997d50)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:49.560840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:49.656249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d7f1ef6383450f800ed392fd997d50', 'request_id': 'req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:49.680220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d7f1ef6383450f800ed392fd997d50', 'request_id': 'req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:49.707510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d7f1ef6383450f800ed392fd997d50', 'request_id': 'req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:49.733290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] The token cache is being invalidate because project b1d7f1ef6383450f800ed392fd997d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:49.733677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:49.734605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:49.737264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 907/1795] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:49 2022] DELETE /identity/v3/projects/b1d7f1ef6383450f800ed392fd997d50 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:49.742552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:49.772602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2c415-5d6e-4df8-87ce-02ea89b763a3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:50.014737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:50.016208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:50.041379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:50.042084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.042387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.042675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] PATH_INFO: `/v3/users/87fc6422c571402099f24adfd96949ef` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.050000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87fc6422c571402099f24adfd96949ef)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:50.053348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:50.227290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87fc6422c571402099f24adfd96949ef'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.241835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87fc6422c571402099f24adfd96949ef'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.245218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49c2a-65bf-4c44-8c6b-5d45b16089eb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:50.257880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:50.258580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.258871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.259154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] PATH_INFO: `/v3/projects/9240a6e5c4ac4cbb8c4eabde1471126c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.265873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87fc6422c571402099f24adfd96949ef'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.267310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9240a6e5c4ac4cbb8c4eabde1471126c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:50.267886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87fc6422c571402099f24adfd96949ef'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.272778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:50.286007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-df154e32-9b4b-4d63-bcd1-f01bfc239c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87fc6422c571402099f24adfd96949ef'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.304873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 889/1796] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:49 2022] DELETE /identity/v3/users/87fc6422c571402099f24adfd96949ef => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:50.465007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9240a6e5c4ac4cbb8c4eabde1471126c', 'request_id': 'req-51bb9a79-0daa-489f-a66a-e6aec39e1af5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.482066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9240a6e5c4ac4cbb8c4eabde1471126c', 'request_id': 'req-51bb9a79-0daa-489f-a66a-e6aec39e1af5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.504225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9240a6e5c4ac4cbb8c4eabde1471126c', 'request_id': 'req-51bb9a79-0daa-489f-a66a-e6aec39e1af5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.514453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:50.519708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] The token cache is being invalidate because project 9240a6e5c4ac4cbb8c4eabde1471126c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:50.520512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.524061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-51bb9a79-0daa-489f-a66a-e6aec39e1af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:50.526632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 908/1797] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:50 2022] DELETE /identity/v3/projects/9240a6e5c4ac4cbb8c4eabde1471126c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:50.532128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.533034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.533573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.556670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:50.557909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:50.581074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:50.632130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 890/1798] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:50.641127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.641416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.641738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.665515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:50.666847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:50.682081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:50.688316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 909/1799] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:50.697284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d528-125e-4ef5-b191-a9cdf90a7865 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:50.704559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:50.705273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.705556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.705831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.706650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:50.707021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:50.709915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:50.734214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 891/1800] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:50.741164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:50.756084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdabb0-5621-4d2e-b9ea-9830de33368b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:50.762320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:50.763144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.763665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.764614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.766020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:50.768418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:50.819106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 910/1801] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:50 2022] POST /identity/v3/projects => generated 369 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:50.819678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:50.831237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb432-b444-4367-bbad-eede672ececb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:50.906449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:50.907319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.907637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.907930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.908592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:50.910514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:50.942519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 892/1802] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:50 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:50.947629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:50.961348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef2b82-9c65-4760-aeaf-d8b363dea052 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:50.966214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:50.966902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:50.967298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:50.967652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:50.968488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:50.970886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:50.984125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 911/1803] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:50.988466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.005776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d22ab-5eec-432f-b3b7-acb849edb0a0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.011531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.012216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.012493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.012761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.013404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.015025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.018793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.021294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.028737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 893/1804] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:39:51.038125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae03e6a-b659-4354-b42a-ea320c076529 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.040275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.043378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.044637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.044923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.045256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] PATH_INFO: `/v3/projects/be706cd0f660452d8725caec41617787` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.046618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be706cd0f660452d8725caec41617787)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.048823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.061678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fcc53-ee0a-4265-acd8-80c0f82b02e8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.067441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdb1f4-68a7-4570-af1e-b78e8362bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.069340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdb1f4-68a7-4570-af1e-b78e8362bd55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.069945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdb1f4-68a7-4570-af1e-b78e8362bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.070392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdb1f4-68a7-4570-af1e-b78e8362bd55 admin admin] PATH_INFO: `/v3/projects/680f66a852504afaadf7b2232e4bf557/users/55b5804181684965b14bffafed3f2a52/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.090661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdb1f4-68a7-4570-af1e-b78e8362bd55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680f66a852504afaadf7b2232e4bf557, user_id=55b5804181684965b14bffafed3f2a52, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.093365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdb1f4-68a7-4570-af1e-b78e8362bd55 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.104723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be706cd0f660452d8725caec41617787', 'request_id': 'req-a8f62fc4-65c9-4d71-bc76-e5af15002c79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:51.108349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 894/1805] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:51 2022] PUT /identity/v3/projects/680f66a852504afaadf7b2232e4bf557/users/55b5804181684965b14bffafed3f2a52/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:51.112160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be706cd0f660452d8725caec41617787', 'request_id': 'req-a8f62fc4-65c9-4d71-bc76-e5af15002c79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:51.119548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.119827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.120132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.121784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be706cd0f660452d8725caec41617787', 'request_id': 'req-a8f62fc4-65c9-4d71-bc76-e5af15002c79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:51.134467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] The token cache is being invalidate because project be706cd0f660452d8725caec41617787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:51.134771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:51.135540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a8f62fc4-65c9-4d71-bc76-e5af15002c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:51.137171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 912/1806] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:51 2022] DELETE /identity/v3/projects/be706cd0f660452d8725caec41617787 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:51.142190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee66173-c380-4c2d-9f8d-98af38db24a5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.142542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee66173-c380-4c2d-9f8d-98af38db24a5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.143240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee66173-c380-4c2d-9f8d-98af38db24a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.151964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] MFA Rules not processed for user `55b5804181684965b14bffafed3f2a52`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:51.153188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.166687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ee66173-c380-4c2d-9f8d-98af38db24a5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:51.167902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ee66173-c380-4c2d-9f8d-98af38db24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.218337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:51.230958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 895/1807] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:51.235455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.237267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.249140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 913/1808] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:51.257536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.257823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.258103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.280647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:51.282095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.290760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 914/1809] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:51.295774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.308563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fc531-d967-4b77-9d13-5a321e9d0ed5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.315284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.315563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c01d0-fcfe-49ad-83ae-9e18b735a78c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.315893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.316249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.316486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.317368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:51.318385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.320008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.321067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632eb7c-ac44-4039-a1fe-b1afc2e6bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.321687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eb7c-ac44-4039-a1fe-b1afc2e6bbc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.321971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eb7c-ac44-4039-a1fe-b1afc2e6bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.322437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eb7c-ac44-4039-a1fe-b1afc2e6bbc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.322886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632eb7c-ac44-4039-a1fe-b1afc2e6bbc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.325135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632eb7c-ac44-4039-a1fe-b1afc2e6bbc1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.338482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 915/1810] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:39:51.350035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.363662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b583056-3c1d-4e3b-bc88-bee3cead9279 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.370679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 896/1811] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:51.372648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027ef55-e9ba-40de-b107-b98381c2e622 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.372763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc83ed5-a49e-48e2-8324-bd9f2c9923cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.373193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027ef55-e9ba-40de-b107-b98381c2e622 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.373359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc83ed5-a49e-48e2-8324-bd9f2c9923cf admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.373567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc83ed5-a49e-48e2-8324-bd9f2c9923cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.373851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc83ed5-a49e-48e2-8324-bd9f2c9923cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.374063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027ef55-e9ba-40de-b107-b98381c2e622 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.374545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc83ed5-a49e-48e2-8324-bd9f2c9923cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.375639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 897/1812] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:51.377216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027ef55-e9ba-40de-b107-b98381c2e622 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.377341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc83ed5-a49e-48e2-8324-bd9f2c9923cf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.417021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 916/1813] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:51.421411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.421701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.421979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.430783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:51.447813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:51.449221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.475008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027ef55-e9ba-40de-b107-b98381c2e622 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.566031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 917/1814] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:51.566965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.633584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.634293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.634586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.634873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.638134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.643835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.665215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722604b-2b74-4697-8b68-e50cd4a076d9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.672029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.672928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.673219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.674301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.676187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.679541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.689717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 898/1815] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/users => generated 290 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:51.692243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.699580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:51.717877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 918/1816] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:51 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:51.718950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.731228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a99f-c639-421f-b7d3-848366d062fd admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.732370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca21c71-08df-4a02-b45f-0d6c5ad6b60a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.740976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.742063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.742360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.742641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.743433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.745559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.755463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.756227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.756517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.756808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.774201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 919/1817] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:51.777604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.789495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.791759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.793003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de716f0-dedc-48f0-af06-74ca4df2eebb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.798337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.798958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.799285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.800336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.801148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.803703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.808872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 899/1818] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:51.820751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 920/1819] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:51.821926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.826057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.833803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04595c0-0c1c-4490-b548-f2b476c2fb7f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.838496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:51.841239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.842586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.843056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.844660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] PATH_INFO: `/v3/projects/7481454b9b424f3595f56f7e4b23cce3/users/8f227062ab814d54a7005971741e34f1/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.866364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dc23e-96fd-45fb-8012-d03b6c57c038 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.872936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:51.873968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:51.874453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:51.875469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:51.876595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.877529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7481454b9b424f3595f56f7e4b23cce3, user_id=8f227062ab814d54a7005971741e34f1, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:51.878975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.884105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:51.894411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 900/1820] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:51.899661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.912595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 921/1821] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:51 2022] PUT /identity/v3/projects/7481454b9b424f3595f56f7e4b23cce3/users/8f227062ab814d54a7005971741e34f1/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:51.917556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:51.920519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7ee03-cf2b-4c2c-9ede-ba3ba0678beb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:51.978882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:52.015416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c645143-5158-46ec-aa24-bce1607d3dc6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.023163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.024381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.024870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.025364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.027118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.029880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.048193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 922/1822] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:52.052352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.063666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c03b-37b5-4a7f-886e-7d4c31de26a8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.071166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f9f14-40cd-4805-ade4-703e2ce4bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.072181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f9f14-40cd-4805-ade4-703e2ce4bb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.072664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f9f14-40cd-4805-ade4-703e2ce4bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.073194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f9f14-40cd-4805-ade4-703e2ce4bb9e admin admin] PATH_INFO: `/v3/projects/7481454b9b424f3595f56f7e4b23cce3/users/8f227062ab814d54a7005971741e34f1/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.075716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f9f14-40cd-4805-ade4-703e2ce4bb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7481454b9b424f3595f56f7e4b23cce3, user_id=8f227062ab814d54a7005971741e34f1, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.080827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f9f14-40cd-4805-ade4-703e2ce4bb9e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.108652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6baa6c-dbbc-4f7c-ad0b-f9923063e309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.109340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6baa6c-dbbc-4f7c-ad0b-f9923063e309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.109616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6baa6c-dbbc-4f7c-ad0b-f9923063e309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.109892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6baa6c-dbbc-4f7c-ad0b-f9923063e309 admin admin] PATH_INFO: `/v3/projects/0217732cd481477381039631f1249998/users/546ff2248ba6414988ebcd451c9f109f/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.117690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 923/1823] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:52 2022] PUT /identity/v3/projects/7481454b9b424f3595f56f7e4b23cce3/users/8f227062ab814d54a7005971741e34f1/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:52.129079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.129079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.129079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.172812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6baa6c-dbbc-4f7c-ad0b-f9923063e309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0217732cd481477381039631f1249998, user_id=546ff2248ba6414988ebcd451c9f109f, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.176340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] MFA Rules not processed for user `8f227062ab814d54a7005971741e34f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:52.178674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.180795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6baa6c-dbbc-4f7c-ad0b-f9923063e309 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.205776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 901/1824] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:51 2022] PUT /identity/v3/projects/0217732cd481477381039631f1249998/users/546ff2248ba6414988ebcd451c9f109f/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:52.218981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.219163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.219537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.270394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] MFA Rules not processed for user `546ff2248ba6414988ebcd451c9f109f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:52.273010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.313161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 924/1825] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:52.328329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.330898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.385052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 902/1826] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:52.390470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.415084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe348-1c2a-4eae-b85c-d6d8c2da7189 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.462488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfce55-d54a-4fdd-9350-5dd0f66aeafe None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.468492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6205b-07b7-4790-b84c-56364453b43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.469244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6205b-07b7-4790-b84c-56364453b43c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.469526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6205b-07b7-4790-b84c-56364453b43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.470174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6205b-07b7-4790-b84c-56364453b43c admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.470658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6205b-07b7-4790-b84c-56364453b43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.472693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6205b-07b7-4790-b84c-56364453b43c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.473025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b14cd-2851-4dce-8045-e4f21b9643c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.474185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b14cd-2851-4dce-8045-e4f21b9643c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.474467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b14cd-2851-4dce-8045-e4f21b9643c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.474741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b14cd-2851-4dce-8045-e4f21b9643c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.484643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:52.500977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:52.512908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 903/1827] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:52 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:52.515061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab1e45-298c-424e-9d46-c737f22263cb None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.515486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab1e45-298c-424e-9d46-c737f22263cb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.515896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab1e45-298c-424e-9d46-c737f22263cb None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.517431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 904/1828] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:52.517986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab1e45-298c-424e-9d46-c737f22263cb None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.553216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b14cd-2851-4dce-8045-e4f21b9643c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.554688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b14cd-2851-4dce-8045-e4f21b9643c6 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.569988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 925/1829] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:52.585307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.585545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.586389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.603590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab1e45-298c-424e-9d46-c737f22263cb None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.618936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.620145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.620465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.620752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.625000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.626202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] MFA Rules not processed for user `e605979196e14110a19a6aff30f2006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:52.627457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.627631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.655396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 905/1830] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:52 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:52.655396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.656205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.668718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf509e9a-a6db-4d61-93ff-de338f0a90ec admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.673419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.674051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.674335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.674607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] PATH_INFO: `/v3/users/a7024ec76baa404db8ce1041302a60bb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.675847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7024ec76baa404db8ce1041302a60bb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.678579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.732709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 926/1831] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:52.733741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.735035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.742055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7024ec76baa404db8ce1041302a60bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:52.752702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7024ec76baa404db8ce1041302a60bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:52.767605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7024ec76baa404db8ce1041302a60bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:52.768358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7024ec76baa404db8ce1041302a60bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:52.777780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7024ec76baa404db8ce1041302a60bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:52.787997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 906/1832] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:52 2022] DELETE /identity/v3/users/a7024ec76baa404db8ce1041302a60bb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:52.788901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.796800 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.810644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5b40f-f753-457b-b7d6-420308ca0d38 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.869235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.870035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.870309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.870898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.872825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.874355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:52.895439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112217d8-d295-4d37-8485-849a7a51b81f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.899595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:52.902189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.902847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.903185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.903452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:52.904187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.906054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.920861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 907/1833] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:52.920861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.922784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:52.953015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:52.954361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:52.967155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 927/1834] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:52.967961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:52.983151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc2323-0552-4e84-8701-4ff0487c9543 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:52.988799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:52.990136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:52.990523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:52.990905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] PATH_INFO: `/v3/projects/fef7ba8780ce41389497d1ac308db545/users/dc14904501e943b59b20e448e5afec5f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.008468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef7ba8780ce41389497d1ac308db545, user_id=dc14904501e943b59b20e448e5afec5f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.011369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.025905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62990133-ff20-4628-bb8b-640edc0c0f10 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:53.031014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 928/1835] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:52 2022] PUT /identity/v3/projects/fef7ba8780ce41389497d1ac308db545/users/dc14904501e943b59b20e448e5afec5f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:53.036770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:53.128303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:53.129074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.129391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.129667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.133023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:53.139520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0db58-7078-4d0c-b96c-62d7c17d75f3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:53.146836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:53.147666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.147845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.148164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.149150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.149890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.150162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32a8d-53ee-4959-bac2-eda2c9a482d2 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.153222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.167650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 929/1836] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:53.172911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:53.178015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 908/1837] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:53.189389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73850089-56ce-4a68-b58f-a38835afd711 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:53.198716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:53.199510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.199777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.200163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] PATH_INFO: `/v3/projects/fef7ba8780ce41389497d1ac308db545/users/dc14904501e943b59b20e448e5afec5f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.202175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef7ba8780ce41389497d1ac308db545, user_id=dc14904501e943b59b20e448e5afec5f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.205039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.232185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.232185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.232185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.233106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 930/1838] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:53 2022] PUT /identity/v3/projects/fef7ba8780ce41389497d1ac308db545/users/dc14904501e943b59b20e448e5afec5f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:53.262795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] MFA Rules not processed for user `dc14904501e943b59b20e448e5afec5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:53.264972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:53.400640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:53.408048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:53.439527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 909/1839] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:53.453390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:53.454873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:53.467294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:53.509268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3ab28-a1ab-464f-bb59-5fe10af088a4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:53.514378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:53.515072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.515355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.515632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] PATH_INFO: `/v3/projects/5b417c71b7714807895bcf1d10fbc1c3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.517794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b417c71b7714807895bcf1d10fbc1c3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.519744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.541197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea48381-cba1-4eb1-b56e-7040ba3a40cd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:53.545160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:53.546111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.546389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.546664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.555142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:53.558613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.560244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.570773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 910/1840] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:53.580571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b417c71b7714807895bcf1d10fbc1c3', 'request_id': 'req-8ea4c9c5-12a4-40be-b65d-157da77197d5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.588952 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b417c71b7714807895bcf1d10fbc1c3', 'request_id': 'req-8ea4c9c5-12a4-40be-b65d-157da77197d5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.596452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b417c71b7714807895bcf1d10fbc1c3', 'request_id': 'req-8ea4c9c5-12a4-40be-b65d-157da77197d5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.608488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] The token cache is being invalidate because project 5b417c71b7714807895bcf1d10fbc1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:53.608800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.609469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.611040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 931/1841] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:53 2022] DELETE /identity/v3/projects/5b417c71b7714807895bcf1d10fbc1c3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:53.614949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:53.625893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046031a-b85d-4d03-b434-8c696c18fce9 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:53.686737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:53.687449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:53.687762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:53.688069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] PATH_INFO: `/v3/users/674166ddd0214e2abf1daaf2920453fc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:53.689345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674166ddd0214e2abf1daaf2920453fc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:53.691288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:53.746470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674166ddd0214e2abf1daaf2920453fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.753926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674166ddd0214e2abf1daaf2920453fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.767314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674166ddd0214e2abf1daaf2920453fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.768330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674166ddd0214e2abf1daaf2920453fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.776609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cab3ee76-1054-44cd-8284-f03351f090d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674166ddd0214e2abf1daaf2920453fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:53.785496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 911/1842] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:53 2022] DELETE /identity/v3/users/674166ddd0214e2abf1daaf2920453fc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:53.988190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:53.990778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.009190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c9c5-12a4-40be-b65d-157da77197d5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.097213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.098238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.098521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.099898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] PATH_INFO: `/v3/projects/b7109d96dd874fa0b4c0399a1ed58519` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.101448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7109d96dd874fa0b4c0399a1ed58519)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.103372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.158002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7109d96dd874fa0b4c0399a1ed58519', 'request_id': 'req-8ded60fc-bfdd-4490-9c7f-ad793306aed8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.164792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7109d96dd874fa0b4c0399a1ed58519', 'request_id': 'req-8ded60fc-bfdd-4490-9c7f-ad793306aed8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.173426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7109d96dd874fa0b4c0399a1ed58519', 'request_id': 'req-8ded60fc-bfdd-4490-9c7f-ad793306aed8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.185045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] The token cache is being invalidate because project b7109d96dd874fa0b4c0399a1ed58519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:54.185401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.186439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.188140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 932/1843] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:53 2022] DELETE /identity/v3/projects/b7109d96dd874fa0b4c0399a1ed58519 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:54.193996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220986-3b57-4beb-84cb-b7833afb0535 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.194209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220986-3b57-4beb-84cb-b7833afb0535 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.194493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56220986-3b57-4beb-84cb-b7833afb0535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.213663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.214174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-56220986-3b57-4beb-84cb-b7833afb0535 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:54.214990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.215368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-56220986-3b57-4beb-84cb-b7833afb0535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.236675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:54.275829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:54.289963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 912/1844] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:54.300236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307aae6-c641-4547-9f08-29db077f83be None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.300524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307aae6-c641-4547-9f08-29db077f83be None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.300797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307aae6-c641-4547-9f08-29db077f83be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.301698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded60fc-bfdd-4490-9c7f-ad793306aed8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.308761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.309603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.309899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.310268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] PATH_INFO: `/v3/users/546ff2248ba6414988ebcd451c9f109f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.311685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546ff2248ba6414988ebcd451c9f109f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.314535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.324759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9307aae6-c641-4547-9f08-29db077f83be None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:54.326698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9307aae6-c641-4547-9f08-29db077f83be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.337366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 913/1845] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:54.341582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307aae6-c641-4547-9f08-29db077f83be None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.354976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307aae6-c641-4547-9f08-29db077f83be None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.361108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.361811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.362575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.362918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.363586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:54.363896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.366179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.370884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546ff2248ba6414988ebcd451c9f109f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.380003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 914/1846] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:39:54.381300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546ff2248ba6414988ebcd451c9f109f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.382150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546ff2248ba6414988ebcd451c9f109f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.384795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.387452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.392462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546ff2248ba6414988ebcd451c9f109f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.403492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56132c28-4818-423f-a213-82514ea0476d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.406878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26b18195-142f-401d-be53-82a7592de859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546ff2248ba6414988ebcd451c9f109f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:54.409249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.409934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.410219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.410498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.411335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.413425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.413526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 933/1847] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:54 2022] DELETE /identity/v3/users/546ff2248ba6414988ebcd451c9f109f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:54.419793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.420120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.420397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.442912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:54.444515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.447993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 915/1848] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:54.451813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.540214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 934/1849] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:54.547392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.549968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.553849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311abb37-175d-4e57-9ef2-526397c12ae4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.559875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.560433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.560780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.561050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.561725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.564978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.597350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 916/1850] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:54.599448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.622004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a937f-bb84-4756-8a4d-5825864b2098 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.633271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.634024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.634307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.634697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.635363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.637128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.650615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 917/1851] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:54.654793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.673766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd360de-3fb6-4733-ba10-0d67a4dbbcc4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.679040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2539-765e-49d9-9572-2d65d6de9974 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.680374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.681343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.681782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.682202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.683003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.684264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443d49c-c8c2-4186-84ba-b4e4ec520bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.685024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.685183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443d49c-c8c2-4186-84ba-b4e4ec520bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.685540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443d49c-c8c2-4186-84ba-b4e4ec520bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.685883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443d49c-c8c2-4186-84ba-b4e4ec520bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.697102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443d49c-c8c2-4186-84ba-b4e4ec520bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.698538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443d49c-c8c2-4186-84ba-b4e4ec520bd9 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.700192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 918/1852] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:54.705572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.710784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 935/1853] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:54.718773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb3686-a219-4bb3-bff7-7e632ab72527 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:54.723871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:54.725134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.725134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.725134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] PATH_INFO: `/v3/projects/35e653077b054b33bb90d5d7918ef4c4/users/2200bb80b6634fb69db1570ef64ff714/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.740246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e653077b054b33bb90d5d7918ef4c4, user_id=2200bb80b6634fb69db1570ef64ff714, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:54.743393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:54.764062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 919/1854] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:54 2022] PUT /identity/v3/projects/35e653077b054b33bb90d5d7918ef4c4/users/2200bb80b6634fb69db1570ef64ff714/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:54.776413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:54.776795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:54.777103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:54.787566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:54.813427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] MFA Rules not processed for user `2200bb80b6634fb69db1570ef64ff714`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:54.814718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:54.912787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 936/1855] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:54.914159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:54.965578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:55.010478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d68dc6-9774-4a8e-81ed-8adc39176579 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.018281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.019773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.020319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.021545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.022646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.027128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.053746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.056746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.078007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee554d-9809-4143-b514-af7c2542f512 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.090740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 920/1856] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:54 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:55.093675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.094343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.094642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.094908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] PATH_INFO: `/v3/projects/0217732cd481477381039631f1249998` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.096395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.101367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0217732cd481477381039631f1249998)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.107929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.200821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59c9e2-a0d8-4bfc-9a2d-90f75e56c933 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.202228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0217732cd481477381039631f1249998', 'request_id': 'req-600928da-ccf3-45f5-af5e-037d3831c970'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.210262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0217732cd481477381039631f1249998', 'request_id': 'req-600928da-ccf3-45f5-af5e-037d3831c970'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.221932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0217732cd481477381039631f1249998', 'request_id': 'req-600928da-ccf3-45f5-af5e-037d3831c970'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.237670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] The token cache is being invalidate because project 0217732cd481477381039631f1249998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:55.238237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.242785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.244809 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 937/1857] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:55 2022] DELETE /identity/v3/projects/0217732cd481477381039631f1249998 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:55.248968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.253042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.280961 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.281607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.281892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.282616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.283253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.286037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.316962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 921/1858] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:55 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:55.323214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.325825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.338370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614f2d8-4dd0-49ac-b136-f7b052ec3633 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.343262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.343585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600928da-ccf3-45f5-af5e-037d3831c970 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.343883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.344292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.344518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.345217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.346814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.348982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.349924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.350206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.350487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] PATH_INFO: `/v3/users/dc14904501e943b59b20e448e5afec5f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.351646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc14904501e943b59b20e448e5afec5f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.353642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.357653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 922/1859] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:39:55.361757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.373043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f37ce-a421-417c-b771-7d391a989da4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.376237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:55.379421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.380926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.381091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.381352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] PATH_INFO: `/v3/projects/5a21603a34314a5cba16d611993f4323/users/2b56fbcc126947c09167cd5702f9cd7f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.398510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a21603a34314a5cba16d611993f4323, user_id=2b56fbcc126947c09167cd5702f9cd7f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.401326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.403735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc14904501e943b59b20e448e5afec5f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.411285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc14904501e943b59b20e448e5afec5f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.411901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc14904501e943b59b20e448e5afec5f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.416144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 923/1860] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:55 2022] PUT /identity/v3/projects/5a21603a34314a5cba16d611993f4323/users/2b56fbcc126947c09167cd5702f9cd7f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:55.419748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc14904501e943b59b20e448e5afec5f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.420308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.422006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.433848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc14904501e943b59b20e448e5afec5f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.440043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 938/1861] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:55 2022] DELETE /identity/v3/users/dc14904501e943b59b20e448e5afec5f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:55.441374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:55.519222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896cf61-034a-443c-a044-de2a210e2526 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.589592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce1671-5b70-4753-90e7-864d0faa2f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.590216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1671-5b70-4753-90e7-864d0faa2f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.590495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1671-5b70-4753-90e7-864d0faa2f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.590770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1671-5b70-4753-90e7-864d0faa2f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.591448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce1671-5b70-4753-90e7-864d0faa2f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.593099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce1671-5b70-4753-90e7-864d0faa2f5b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.608036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 924/1862] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:39:55.609625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.621630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e92e0-027b-4920-bb42-fd853d07f39d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.627543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.628750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.629038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.629322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] PATH_INFO: `/v3/projects/5a21603a34314a5cba16d611993f4323/users/2b56fbcc126947c09167cd5702f9cd7f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.633332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a21603a34314a5cba16d611993f4323, user_id=2b56fbcc126947c09167cd5702f9cd7f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.636706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.648821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 939/1863] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:55 2022] PUT /identity/v3/projects/5a21603a34314a5cba16d611993f4323/users/2b56fbcc126947c09167cd5702f9cd7f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:55.659029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.659319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.659597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.690460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] MFA Rules not processed for user `2b56fbcc126947c09167cd5702f9cd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:55.692005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.780306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.783249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.800851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 925/1864] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:55.826800 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:55.828499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.874277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e995-2dd2-42a2-95ec-090db0f60acf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.883248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:55.886923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.887570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.887850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.888150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] PATH_INFO: `/v3/projects/fef7ba8780ce41389497d1ac308db545` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.889280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef7ba8780ce41389497d1ac308db545)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.891175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.899574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6db9f-779c-45ba-81b8-ba0c6227be55 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:55.904736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:55.905463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.905749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.906020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.907852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:55.941322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef7ba8780ce41389497d1ac308db545', 'request_id': 'req-bdb5830f-62b6-4068-a557-e574d82ed175'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.948647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef7ba8780ce41389497d1ac308db545', 'request_id': 'req-bdb5830f-62b6-4068-a557-e574d82ed175'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.956694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef7ba8780ce41389497d1ac308db545', 'request_id': 'req-bdb5830f-62b6-4068-a557-e574d82ed175'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.967640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] The token cache is being invalidate because project fef7ba8780ce41389497d1ac308db545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:55.967924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.968607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bdb5830f-62b6-4068-a557-e574d82ed175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:55.970118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 940/1865] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:55 2022] DELETE /identity/v3/projects/fef7ba8780ce41389497d1ac308db545 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:55.974555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:55.975798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:55.976129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272649f-c1cf-4753-9985-f3d7b1a9bef7 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:55.976230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:55.976469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:55.985271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 926/1866] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:55.996078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:55.997318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:56.006385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:56.066035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 941/1867] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:56.073781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.074071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.074354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.112156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:56.112156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:56.117383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 927/1868] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:56.121257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.131454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3639-38fe-4c23-be73-11b09209b5b8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.137525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.138623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.138876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.139198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.140031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:56.140464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.142059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.156428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 942/1869] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:56.160597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.174779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c344703-1896-47dc-8376-4a7df3d5e06f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.180257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.180873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.181153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.181471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.182051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.183928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.223221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 928/1870] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:56 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:56.229044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.240826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541ec6c-a8a5-4d17-a036-938b4a09eb20 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.364289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.365347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.365635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.365914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.366572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.369882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.409245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 943/1871] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:56 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:56.415539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.428701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff898e-3192-41a0-a593-dffd1c9f95b8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.433468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.434070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.434345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.434612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.435256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.437065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.452881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 929/1872] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:56.453838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.465701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab051157-273b-4641-83fb-c7cdd01cc870 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.470808 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305b23-ac43-4f36-bb66-7d5284ff07c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.471156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305b23-ac43-4f36-bb66-7d5284ff07c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.471436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305b23-ac43-4f36-bb66-7d5284ff07c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.471712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305b23-ac43-4f36-bb66-7d5284ff07c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.472398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305b23-ac43-4f36-bb66-7d5284ff07c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.475380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305b23-ac43-4f36-bb66-7d5284ff07c9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.492607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 944/1873] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:56.493442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.508146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cc83-cee9-494a-939b-9e75a216457c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.513522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.514277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.514667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.515038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] PATH_INFO: `/v3/projects/a0703401286f4528948eb901934de0c4/users/ef59e4477467484080a04bfc8cc67387/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.531834 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0703401286f4528948eb901934de0c4, user_id=ef59e4477467484080a04bfc8cc67387, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.534644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.550621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 930/1874] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:56 2022] PUT /identity/v3/projects/a0703401286f4528948eb901934de0c4/users/ef59e4477467484080a04bfc8cc67387/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:56.563355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.563713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.564027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.602927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] MFA Rules not processed for user `ef59e4477467484080a04bfc8cc67387`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:56.604184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:56.684050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 945/1875] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:56.685265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.746598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.748082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:56.761218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:56.761688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc278d-956d-4a0f-870d-e3e0fb0e9c65 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.766755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.767391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.767679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.768020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.768629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.770580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.824997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 931/1876] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:56 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:56.828944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.834528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d307d7-b4f1-408c-96b9-fbb736591064 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.862047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:56.903280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.903922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.905933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.905933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] PATH_INFO: `/v3/users/55b5804181684965b14bffafed3f2a52` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.906463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b5804181684965b14bffafed3f2a52)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.906786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c7cc3-e8aa-46d0-b091-6abbf58796db admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.908468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.912526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.913407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.913692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.914100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.914821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.917063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.944951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 932/1877] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:56 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:56.949402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.961180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4ca41-6dd2-4252-9c39-3de2b8b6301e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:56.966181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:56.966988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:56.967171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b5804181684965b14bffafed3f2a52'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:56.967458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:56.967830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:56.968611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:56.970433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:56.978165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b5804181684965b14bffafed3f2a52'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:56.978918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b5804181684965b14bffafed3f2a52'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:56.982603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 933/1878] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:56.987099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:56.987257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b5804181684965b14bffafed3f2a52'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:56.990694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.001398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-38a679ce-d59f-4be7-b0f9-7bc2e40365d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b5804181684965b14bffafed3f2a52'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.004463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2147c40-159b-46e6-b599-c8d0572c17f3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.009747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.010284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 946/1879] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:56 2022] DELETE /identity/v3/users/55b5804181684965b14bffafed3f2a52 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:57.012352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.013116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.013116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] PATH_INFO: `/v3/projects/1e1b4fe9aa794255a742bc9570766367/users/9f517c0c28504ef5a24116c53b734080/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.014726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.015040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.015318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.040912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1b4fe9aa794255a742bc9570766367, user_id=9f517c0c28504ef5a24116c53b734080, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.042797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:57.044055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.044271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.061867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 934/1880] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:56 2022] PUT /identity/v3/projects/1e1b4fe9aa794255a742bc9570766367/users/9f517c0c28504ef5a24116c53b734080/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:57.065533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:57.086250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:57.127808 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 947/1881] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:57.136297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b8c53-f195-4ed6-84d0-67b39f4a7ea7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.138252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:57.139704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.141316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.141926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.142211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.142486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.143134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.144772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.156758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 935/1882] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:57.160815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:57.170926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be86efd-430c-4bef-ae58-4cf7535125e6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.175631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7440601-fa82-4b61-93ef-a455230aa317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.176300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7440601-fa82-4b61-93ef-a455230aa317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.176689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7440601-fa82-4b61-93ef-a455230aa317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.176960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7440601-fa82-4b61-93ef-a455230aa317 admin admin] PATH_INFO: `/v3/projects/1e1b4fe9aa794255a742bc9570766367/users/9f517c0c28504ef5a24116c53b734080/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.178881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7440601-fa82-4b61-93ef-a455230aa317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1b4fe9aa794255a742bc9570766367, user_id=9f517c0c28504ef5a24116c53b734080, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.181552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7440601-fa82-4b61-93ef-a455230aa317 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.191864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 936/1883] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:57 2022] PUT /identity/v3/projects/1e1b4fe9aa794255a742bc9570766367/users/9f517c0c28504ef5a24116c53b734080/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:57.201008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7c55b-07b6-4e50-a4db-44d153c9e1f8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.203528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.203854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.204171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.232742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] MFA Rules not processed for user `9f517c0c28504ef5a24116c53b734080`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:57.233960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.252135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.252746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.253027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.253299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.281287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:57.307778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 937/1884] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:57.315420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.316891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.319280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:57.320692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.327324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 948/1885] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:57.381825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7cce8-e69b-49b6-99fe-0a30469b2eae None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.386286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.386874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.387163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.387596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.389502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.466911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.468554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.480177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 938/1886] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:39:57.668266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:57.670324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:57.700311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3048694-ab4d-45f7-a3bf-a3e02e133376 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.707752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.708798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.709082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.709364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] PATH_INFO: `/v3/projects/680f66a852504afaadf7b2232e4bf557` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.710556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=680f66a852504afaadf7b2232e4bf557)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.712683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.761452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:57.772027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680f66a852504afaadf7b2232e4bf557', 'request_id': 'req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.778575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680f66a852504afaadf7b2232e4bf557', 'request_id': 'req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.786597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680f66a852504afaadf7b2232e4bf557', 'request_id': 'req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.798350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] The token cache is being invalidate because project 680f66a852504afaadf7b2232e4bf557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:57.798896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.799682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.801528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 949/1887] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:57 2022] DELETE /identity/v3/projects/680f66a852504afaadf7b2232e4bf557 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:57.806729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:57.817891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0feb-ff34-46b5-8150-3956eee4ac0e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:57.891364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:57.892529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:57.896439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:57.896922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] PATH_INFO: `/v3/users/8f227062ab814d54a7005971741e34f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:57.898904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f227062ab814d54a7005971741e34f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:57.901853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:57.970501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:57.979397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f227062ab814d54a7005971741e34f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:57.989006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f227062ab814d54a7005971741e34f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:58.005642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f227062ab814d54a7005971741e34f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:58.006547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f227062ab814d54a7005971741e34f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:58.013705 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf4e8bc8-1688-4f9f-bbe1-ff29cf74bfa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f227062ab814d54a7005971741e34f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:58.020024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 939/1888] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:57 2022] DELETE /identity/v3/users/8f227062ab814d54a7005971741e34f1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:39:58.826335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:58.829256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:58.873777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb9afe-62ab-43fe-afb6-d6f000c240ea admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:58.946796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:58.947400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:58.948147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:58.948430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] PATH_INFO: `/v3/projects/7481454b9b424f3595f56f7e4b23cce3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:58.949805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7481454b9b424f3595f56f7e4b23cce3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:58.951855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.041877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7481454b9b424f3595f56f7e4b23cce3', 'request_id': 'req-6849aa1d-3add-46fb-8054-215033e601ef'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.068798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7481454b9b424f3595f56f7e4b23cce3', 'request_id': 'req-6849aa1d-3add-46fb-8054-215033e601ef'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.089856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7481454b9b424f3595f56f7e4b23cce3', 'request_id': 'req-6849aa1d-3add-46fb-8054-215033e601ef'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.116664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] The token cache is being invalidate because project 7481454b9b424f3595f56f7e4b23cce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:39:59.116961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.118395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6849aa1d-3add-46fb-8054-215033e601ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.120826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 950/1889] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:39:58 2022] DELETE /identity/v3/projects/7481454b9b424f3595f56f7e4b23cce3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:59.126838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.127700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.128172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.169623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:59.172133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.287607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 940/1890] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:59.295222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.295786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.296081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.333290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:59.334585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.346212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 951/1891] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:59.352302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.364820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f780bd-933a-40cb-b9d3-cb8480c1650d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.371321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.372234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.372517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.372791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.373455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:39:59.373821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.382112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.398312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 941/1892] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:39:59.405829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.416772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350eafa-33f3-463e-9593-844bf3b45338 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.423051 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.423669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.423969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.424527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.424980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.439642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.487021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 952/1893] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:59.488548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.514711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a243b-ac76-4e8c-b799-9dd748d5dbb4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.591146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.591803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.592253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.592536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.593190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.595098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.624090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 942/1894] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:39:59.625507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.637227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c727861-b3dd-43b9-9435-d776ed46a5c4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.643262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.643977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.644337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.645044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.645803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.647604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.662434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.664194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.666965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 953/1895] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:59.668020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.676434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce2eb-9d50-4d7e-818e-afa7ba1ff513 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.681994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.682635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.682865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4325943-f496-4821-9852-412b73b9477a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.682966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.683186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] PATH_INFO: `/v3/users/ec2024714f744b8fabc1a2cc4dd4c4fd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.684331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2024714f744b8fabc1a2cc4dd4c4fd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.686257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.687873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.688574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.688943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.689290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.690007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.691679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.702630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 954/1896] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:59.706805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.713206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:59.717013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:59.717586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be45af-f285-4b63-8dd3-d29538620967 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.724204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398afa81-4964-4d20-83ee-291a9a810182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.725124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398afa81-4964-4d20-83ee-291a9a810182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.725497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398afa81-4964-4d20-83ee-291a9a810182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.725910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398afa81-4964-4d20-83ee-291a9a810182 admin admin] PATH_INFO: `/v3/projects/a80b6dceb9c04c1bbdaa91f0c23cf535/users/3fe3e9df713a431cb12a7b9fe9ce3c72/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.736514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2024714f744b8fabc1a2cc4dd4c4fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.742565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398afa81-4964-4d20-83ee-291a9a810182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80b6dceb9c04c1bbdaa91f0c23cf535, user_id=3fe3e9df713a431cb12a7b9fe9ce3c72, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.744490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2024714f744b8fabc1a2cc4dd4c4fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.745293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398afa81-4964-4d20-83ee-291a9a810182 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.758289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2024714f744b8fabc1a2cc4dd4c4fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.759146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2024714f744b8fabc1a2cc4dd4c4fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.759260 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 955/1897] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:39:59 2022] PUT /identity/v3/projects/a80b6dceb9c04c1bbdaa91f0c23cf535/users/3fe3e9df713a431cb12a7b9fe9ce3c72/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:59.766091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-353a2e7a-0056-4c2c-afb9-2b06d18cb58d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2024714f744b8fabc1a2cc4dd4c4fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:39:59.770856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.771227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.771515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.774102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 943/1898] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:39:59 2022] DELETE /identity/v3/users/ec2024714f744b8fabc1a2cc4dd4c4fd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:39:59.777589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.777879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.778159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.797083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:59.798290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.802631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] MFA Rules not processed for user `3fe3e9df713a431cb12a7b9fe9ce3c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:59.803824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.859399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 944/1899] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:59.869024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 956/1900] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:39:59.872988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.872988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.873515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.874859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:59.888631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1236a2-18ab-45e9-86f4-576562c50785 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.893539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.894165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.894446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.894933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.895386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:59.897348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:59.925234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 957/1901] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:39:59.929785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:59.931017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f55d4-1e94-4700-bd33-006d85bcc541 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:59.980788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:59.981401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:59.981703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:59.981977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:59.995392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:59.995546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f120523-c14d-4929-a5c4-08905fcb108f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.000901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.002001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.002001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.002252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.002766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.004746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.025834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 958/1902] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:39:59 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:00.030032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.041256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e4d8e-0474-4061-9164-f33e0ceb0df7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.046377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.047017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.047330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.047613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.048349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.050017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.066307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.066414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 959/1903] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:00.066500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.067748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.075896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 945/1904] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:00.078516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d9f39-1afa-4d82-9725-03afba5b9897 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.084482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.085235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.085523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.085879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] PATH_INFO: `/v3/projects/d4b608c928044acebe13af933e827077/users/cd0f2ad431104203b7604e158dd35ba4/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.106206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b608c928044acebe13af933e827077, user_id=cd0f2ad431104203b7604e158dd35ba4, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.109705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.148678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 960/1905] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:00 2022] PUT /identity/v3/projects/d4b608c928044acebe13af933e827077/users/cd0f2ad431104203b7604e158dd35ba4/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:00.153184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.168435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b264ca-910d-48dd-9cff-bff34f64fe1c service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.289941 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0919e4c-90fd-44bb-9d17-c0f58c8e78e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.290825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0919e4c-90fd-44bb-9d17-c0f58c8e78e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.291302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0919e4c-90fd-44bb-9d17-c0f58c8e78e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.291797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0919e4c-90fd-44bb-9d17-c0f58c8e78e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.292652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0919e4c-90fd-44bb-9d17-c0f58c8e78e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.294619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0919e4c-90fd-44bb-9d17-c0f58c8e78e1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.307338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 946/1906] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:00.311597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.318001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:00.324886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ebb8f-a228-4ccd-8ba3-6723ac2aede9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.339699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.340552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.340938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.341727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] PATH_INFO: `/v3/projects/d4b608c928044acebe13af933e827077/users/cd0f2ad431104203b7604e158dd35ba4/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.343834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b608c928044acebe13af933e827077, user_id=cd0f2ad431104203b7604e158dd35ba4, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.347084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.362554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 961/1907] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:00 2022] PUT /identity/v3/projects/d4b608c928044acebe13af933e827077/users/cd0f2ad431104203b7604e158dd35ba4/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:00.374234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.374624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.375140 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.413455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] MFA Rules not processed for user `cd0f2ad431104203b7604e158dd35ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:00.414521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:00.486713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.488471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:00.516440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 947/1908] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:00.540067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.541930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:00.575269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd7ae-236d-4fa4-8c03-4aaf9703a1a6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.592238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.592891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.593174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.593447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] PATH_INFO: `/v3/projects/d304aba7e3aa44ab9f9fdc1a5ea4049e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.594617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d304aba7e3aa44ab9f9fdc1a5ea4049e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.596592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.597266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:00.621981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8177623-6a73-4b0a-a274-3827b2c590b3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.625958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.626572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.626846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.627114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.637445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.638984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.646823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd304aba7e3aa44ab9f9fdc1a5ea4049e', 'request_id': 'req-a446f36c-46a8-4f93-bfcb-2da44b3f502f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.649095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 948/1909] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:00.653070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd304aba7e3aa44ab9f9fdc1a5ea4049e', 'request_id': 'req-a446f36c-46a8-4f93-bfcb-2da44b3f502f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.660509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd304aba7e3aa44ab9f9fdc1a5ea4049e', 'request_id': 'req-a446f36c-46a8-4f93-bfcb-2da44b3f502f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.672103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] The token cache is being invalidate because project d304aba7e3aa44ab9f9fdc1a5ea4049e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:00.672103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.672925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.674765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 962/1910] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:00 2022] DELETE /identity/v3/projects/d304aba7e3aa44ab9f9fdc1a5ea4049e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:00.679751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:00.691109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab72c-ba84-4e52-be70-b0901a1f3022 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:00.758788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:00.760162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:00.760395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:00.760782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] PATH_INFO: `/v3/users/43a69e9ada4641d7869ae7cc11fdb1c7` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:00.762330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a69e9ada4641d7869ae7cc11fdb1c7)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:00.764670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:00.820960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a69e9ada4641d7869ae7cc11fdb1c7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.830880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a69e9ada4641d7869ae7cc11fdb1c7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.846641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a69e9ada4641d7869ae7cc11fdb1c7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.847416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a69e9ada4641d7869ae7cc11fdb1c7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.856145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a69e9ada4641d7869ae7cc11fdb1c7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:00.873245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 949/1911] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:00 2022] DELETE /identity/v3/users/43a69e9ada4641d7869ae7cc11fdb1c7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:01.022748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.024280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.104638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446f36c-46a8-4f93-bfcb-2da44b3f502f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.130599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:01.165239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.167035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.169265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd0f2ad431104203b7604e158dd35ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b608c928044acebe13af933e827077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.169990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.170279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.170559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.171275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:01.171587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.173277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.188136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 963/1912] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:01.191076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.194122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.206363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d07af-d73f-4b7e-bfa2-1ed0acb105f9 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.211582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd0f2ad431104203b7604e158dd35ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b608c928044acebe13af933e827077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.212219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.212502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.212790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.213444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.215334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.244117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 964/1913] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:01 2022] POST /identity/v3/projects => generated 354 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:01.248170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:01.248292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.256418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84000a82-8785-43d6-a3ad-b16c1d8513d2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.319782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.320458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.320760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.321060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] PATH_INFO: `/v3/projects/9c6f7627090142678484fbb32b42a834` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.322240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6f7627090142678484fbb32b42a834)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.323611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927fa09-e84e-45c5-b68b-2c61fbf64ce3 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.324251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.328865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd0f2ad431104203b7604e158dd35ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b608c928044acebe13af933e827077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.329496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.329777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.330054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.330703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:01.331435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.332668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.344839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 965/1914] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:40:01.381794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6f7627090142678484fbb32b42a834', 'request_id': 'req-6f094da0-f6f4-4c96-b596-498f23cddd2b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.388880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.393906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6f7627090142678484fbb32b42a834', 'request_id': 'req-6f094da0-f6f4-4c96-b596-498f23cddd2b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.400213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fe9ef-31ff-4729-936c-9e01b96777e8 tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.400451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6f7627090142678484fbb32b42a834', 'request_id': 'req-6f094da0-f6f4-4c96-b596-498f23cddd2b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.415540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] The token cache is being invalidate because project 9c6f7627090142678484fbb32b42a834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:01.415825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.416491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6f094da0-f6f4-4c96-b596-498f23cddd2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.418515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 950/1915] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:01 2022] DELETE /identity/v3/projects/9c6f7627090142678484fbb32b42a834 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:01.423010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9cdf-3828-4df8-9424-0f09d78fdea9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.423293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9cdf-3828-4df8-9424-0f09d78fdea9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.423568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9cdf-3828-4df8-9424-0f09d78fdea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.447027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2dfd9cdf-3828-4df8-9424-0f09d78fdea9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:01.448359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2dfd9cdf-3828-4df8-9424-0f09d78fdea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.470059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd0f2ad431104203b7604e158dd35ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4b608c928044acebe13af933e827077', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.470824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.472802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.472802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] PATH_INFO: `/v3/projects/fd834dbc715147f0bbd5024094aa5668` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.472802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fd834dbc715147f0bbd5024094aa5668)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.475175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.519564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:01.530562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 951/1916] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:01.532990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd834dbc715147f0bbd5024094aa5668', 'request_id': 'req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.539064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.539349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.539622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.541056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd834dbc715147f0bbd5024094aa5668', 'request_id': 'req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.549319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd834dbc715147f0bbd5024094aa5668', 'request_id': 'req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.562171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:01.562895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] The token cache is being invalidate because project fd834dbc715147f0bbd5024094aa5668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:01.563198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.563379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.563971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.565543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 966/1917] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:01 2022] DELETE /identity/v3/projects/fd834dbc715147f0bbd5024094aa5668 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:01.597039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:01.646589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 952/1918] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:01.647893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.662527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d89f88-a37b-4a9c-a4ab-f91c18e769ef tempest-VolumeQuotaClassesTest-579039621 tempest-VolumeQuotaClassesTest-579039621-project-admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.668555 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.669306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.669905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.670333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.672932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:01.673376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.675666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.694396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 967/1919] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:01.698597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.709693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea386e-3a33-4f2a-812e-ce5263ab484d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.714960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.715573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.715851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.716149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.716797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.718744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.764772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 953/1920] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:01 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:01.768847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.778751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9bad3-867b-4e4f-b292-23a46afda53c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.816066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.818127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:01.845609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.846234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.846516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.846791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.847417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.850177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.871602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 968/1921] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:01 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:01.875727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.886958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2892193-bed8-435f-8d70-8a6286234d90 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.887333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a35dcd-a400-4628-bf4c-543e77482549 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.892092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.892214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.892689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.892785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.892992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.893095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.893271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] PATH_INFO: `/v3/users/3fe3e9df713a431cb12a7b9fe9ce3c72` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.894887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.894887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.896085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe3e9df713a431cb12a7b9fe9ce3c72)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.896570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.898001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.908816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 969/1922] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:01.914003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.924442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ee536-6039-4a4d-a8f6-d8054724f712 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.931343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.932116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.932524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.932698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.933326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.934937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:01.951678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 970/1923] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:01.955793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:01.957356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe3e9df713a431cb12a7b9fe9ce3c72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.967825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe3e9df713a431cb12a7b9fe9ce3c72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.969041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f91b2-76e0-4aa8-855c-211251cd284f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:01.975022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4424f-4fc7-4dc2-b4a5-127f5e4b7ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:01.975762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4424f-4fc7-4dc2-b4a5-127f5e4b7ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:01.976137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4424f-4fc7-4dc2-b4a5-127f5e4b7ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:01.976498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4424f-4fc7-4dc2-b4a5-127f5e4b7ad2 admin admin] PATH_INFO: `/v3/projects/3d25b0fef6844b24bbee39714b22c228/users/eaad4077d09f4ee8bd3460224e1b443f/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:01.989708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe3e9df713a431cb12a7b9fe9ce3c72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.990774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe3e9df713a431cb12a7b9fe9ce3c72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:01.993349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4424f-4fc7-4dc2-b4a5-127f5e4b7ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d25b0fef6844b24bbee39714b22c228, user_id=eaad4077d09f4ee8bd3460224e1b443f, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:01.996267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4424f-4fc7-4dc2-b4a5-127f5e4b7ad2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.008516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a1910e9d-236a-4042-ab13-e96ed92721c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe3e9df713a431cb12a7b9fe9ce3c72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:02.017344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 954/1924] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:01 2022] DELETE /identity/v3/users/3fe3e9df713a431cb12a7b9fe9ce3c72 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:02.018032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 971/1925] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:01 2022] PUT /identity/v3/projects/3d25b0fef6844b24bbee39714b22c228/users/eaad4077d09f4ee8bd3460224e1b443f/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:02.022620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.023385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.023721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.037631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.038139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.038215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.047426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:02.048674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.074486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] MFA Rules not processed for user `eaad4077d09f4ee8bd3460224e1b443f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:02.075709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.077758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:02.121402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:02.125757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 955/1926] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:02.133843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.135448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.150650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 972/1927] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:02.155453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.156838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.171034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0be61-5035-4a5a-b60a-ebfd8962df23 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.176024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.177273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.177570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.177826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.178455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.180364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.197223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa80ae9-46db-453a-83d0-9cb532fa7aaa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.201625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.202374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.202650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.202975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.209494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 973/1928] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:02 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:02.212143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.213388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.213624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.225236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 956/1929] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:02.290724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b79fa4-06b4-4cb2-836e-73a496e57958 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.295894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.296683 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.297041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.297412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.298136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.300289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.331726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 974/1930] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:02 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:02.338207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.354984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddb436-1e81-4e3f-a484-d6a2ecef866d service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.366210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.370613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.372414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.373336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.374403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.376887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.395329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 957/1931] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:02.401957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.422971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ed05-afac-491c-b271-8de94c472e01 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.433637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432a031-e9e8-499f-817a-fef4e6494cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.435439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432a031-e9e8-499f-817a-fef4e6494cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.436335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432a031-e9e8-499f-817a-fef4e6494cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.436830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432a031-e9e8-499f-817a-fef4e6494cd3 admin admin] PATH_INFO: `/v3/projects/1284bf1ffb05415f80079d778b4b66f0/users/95e3ff1928fd441a95c3a21dd9ec12d9/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.460109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432a031-e9e8-499f-817a-fef4e6494cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1284bf1ffb05415f80079d778b4b66f0, user_id=95e3ff1928fd441a95c3a21dd9ec12d9, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.464570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432a031-e9e8-499f-817a-fef4e6494cd3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.484901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 975/1932] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:02 2022] PUT /identity/v3/projects/1284bf1ffb05415f80079d778b4b66f0/users/95e3ff1928fd441a95c3a21dd9ec12d9/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:02.489095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.507238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b810-f56d-4adf-afaf-20a9f5baa5ad admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.609922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da77af-ac3a-4ea7-a585-462f34f0e84b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.610571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da77af-ac3a-4ea7-a585-462f34f0e84b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.611146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da77af-ac3a-4ea7-a585-462f34f0e84b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.612631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 976/1933] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:02.617891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.619019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.619415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.620039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.620826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.624220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.631066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.631426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.631782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.641736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 958/1934] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:02.646334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:02.667336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:02.671389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e8a9-f8c7-4505-9764-074487ecef02 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:02.673434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.685974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a70df-2bf2-4f92-8eee-53b7b3f29530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:02.686663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a70df-2bf2-4f92-8eee-53b7b3f29530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.686944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a70df-2bf2-4f92-8eee-53b7b3f29530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.687271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a70df-2bf2-4f92-8eee-53b7b3f29530 admin admin] PATH_INFO: `/v3/projects/1284bf1ffb05415f80079d778b4b66f0/users/95e3ff1928fd441a95c3a21dd9ec12d9/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.694578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a70df-2bf2-4f92-8eee-53b7b3f29530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1284bf1ffb05415f80079d778b4b66f0, user_id=95e3ff1928fd441a95c3a21dd9ec12d9, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:02.697339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a70df-2bf2-4f92-8eee-53b7b3f29530 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:02.735980 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 959/1935] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:02 2022] PUT /identity/v3/projects/1284bf1ffb05415f80079d778b4b66f0/users/95e3ff1928fd441a95c3a21dd9ec12d9/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:02.742849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:02.743137 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:02.743416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:02.796577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] MFA Rules not processed for user `95e3ff1928fd441a95c3a21dd9ec12d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:02.797815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:02.855960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 977/1936] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:02.886590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:02.994715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 960/1937] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:03.010774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:03.012818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:03.048912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e743-961f-45ca-abb8-b5219a2effa5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:03.060282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:03.061789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:03.102595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:03.185182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0bd46-fe4b-4d87-aff1-c8dfd77c68db None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:03.189597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:03.190353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:03.190640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:03.190923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:03.194762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:03.195876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:03.196478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:03.197075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:03.210960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:03.213311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:03.214550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:03.215842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:03.233741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 961/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:03.237272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 978/1939] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:03.244504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:03.250109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:03.415534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cdd09-b12d-4562-9ec6-a106b845c605 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:03.435105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:03.435820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:03.436139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:03.436423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] PATH_INFO: `/v3/projects/a80b6dceb9c04c1bbdaa91f0c23cf535` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:03.437702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:03.438343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80b6dceb9c04c1bbdaa91f0c23cf535)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:03.440250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:03.453595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307edba6-7609-4ec5-86db-99bc31cc0251 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:03.458849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:03.462004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:03.462004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:03.462004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:03.535309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80b6dceb9c04c1bbdaa91f0c23cf535', 'request_id': 'req-75799c8b-e1e7-464c-a41c-8bb4437a281b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.543869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80b6dceb9c04c1bbdaa91f0c23cf535', 'request_id': 'req-75799c8b-e1e7-464c-a41c-8bb4437a281b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.545029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:03.546660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:03.550455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80b6dceb9c04c1bbdaa91f0c23cf535', 'request_id': 'req-75799c8b-e1e7-464c-a41c-8bb4437a281b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.557137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 979/1940] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:03.558203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:03.559668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:03.564067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] The token cache is being invalidate because project a80b6dceb9c04c1bbdaa91f0c23cf535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:03.564430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.565114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.566995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 962/1941] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:03 2022] DELETE /identity/v3/projects/a80b6dceb9c04c1bbdaa91f0c23cf535 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:03.570829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:03.572285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:03.577826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4373d-2859-4ba2-8191-555d9098402c service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:03.579672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:03.638069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:03.638762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:03.639112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:03.639392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:03.641168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:03.658321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799c8b-e1e7-464c-a41c-8bb4437a281b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:03.666217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:03.667317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:03.667597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:03.667870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] PATH_INFO: `/v3/users/cd0f2ad431104203b7604e158dd35ba4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:03.670123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd0f2ad431104203b7604e158dd35ba4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:03.672065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:03.712334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:03.729400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd0f2ad431104203b7604e158dd35ba4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.731792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:03.733340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:03.737657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd0f2ad431104203b7604e158dd35ba4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.745527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 980/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:03.751672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd0f2ad431104203b7604e158dd35ba4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.752475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd0f2ad431104203b7604e158dd35ba4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.762148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1aa77371-3211-42f1-97e0-053c87599658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd0f2ad431104203b7604e158dd35ba4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:03.768616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 963/1943] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:03 2022] DELETE /identity/v3/users/cd0f2ad431104203b7604e158dd35ba4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:04.372852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:04.387028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265dbbf-5504-4d31-8b93-ee706e4cbfd9 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:04.479410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:04.480195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.482215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.482682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] PATH_INFO: `/v3/projects/d4b608c928044acebe13af933e827077` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.484867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b608c928044acebe13af933e827077)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:04.487600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:04.591649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b608c928044acebe13af933e827077', 'request_id': 'req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:04.613124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b608c928044acebe13af933e827077', 'request_id': 'req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:04.638927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b608c928044acebe13af933e827077', 'request_id': 'req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:04.678823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] The token cache is being invalidate because project d4b608c928044acebe13af933e827077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:04.680315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:04.684378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:04.689114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 981/1944] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:04 2022] DELETE /identity/v3/projects/d4b608c928044acebe13af933e827077 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:04.694685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c25b2-c811-4c01-8aa1-5355232cd6b9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.695546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c25b2-c811-4c01-8aa1-5355232cd6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.696051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c25b2-c811-4c01-8aa1-5355232cd6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.730182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-699c25b2-c811-4c01-8aa1-5355232cd6b9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:04.732269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-699c25b2-c811-4c01-8aa1-5355232cd6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.759676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:04.762042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.829162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 964/1945] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:04.843599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.843884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.844186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.870047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:04.871526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.883370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 965/1946] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:04.885820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:04.896811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00bf98-d0f8-4834-9436-7f83f1f26edb None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:04.902139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:04.902774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.903054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.903328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.904039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:04.904559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:04.907159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:04.911724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1d5a-d6e7-41d5-a420-f8a77ad8a037 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:04.925887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 966/1947] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:40:04.927062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:04.929255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:04.930149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.930602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.931053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.933363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:04.935103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:04.939670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a7172-0154-452a-a104-781708142156 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:04.949080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:04.949697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:04.950178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:04.951335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:04.952159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:04.954908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:04.985177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:04.994985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 967/1948] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:04 2022] POST /identity/v3/projects => generated 405 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:04.998644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.025828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.028175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.043994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 982/1949] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:05.088179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1875b-106f-4433-954b-55af9959a00d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.093876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.094635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.094932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.095341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.096379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.099558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.123605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 968/1950] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:04 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:05.127174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.138282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326a226-64f6-4fb8-8d5e-c88a1f4f6c74 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.144107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.144711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.144990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.146422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.146736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.148352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.163081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 983/1951] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:05.165649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.177030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281d46-ece7-495d-92d7-31b7e3ef513c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.182494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf620318-cf7d-4173-959c-81114e3aa66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.183585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf620318-cf7d-4173-959c-81114e3aa66f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.183933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf620318-cf7d-4173-959c-81114e3aa66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.184248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf620318-cf7d-4173-959c-81114e3aa66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.184898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf620318-cf7d-4173-959c-81114e3aa66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.186709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf620318-cf7d-4173-959c-81114e3aa66f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.202689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 969/1952] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:05.204649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.218522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e14ca-246c-402c-8515-356961e6a55c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.225648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.226303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.226583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.226859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] PATH_INFO: `/v3/projects/37a895bf3cac4122bc1c7ead37cb152c/users/6803aea24b3c472e914c671f8fbba518/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.243381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a895bf3cac4122bc1c7ead37cb152c, user_id=6803aea24b3c472e914c671f8fbba518, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.246245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.262987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 984/1953] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:05 2022] PUT /identity/v3/projects/37a895bf3cac4122bc1c7ead37cb152c/users/6803aea24b3c472e914c671f8fbba518/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:05.274208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.274680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.275096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.319490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] MFA Rules not processed for user `6803aea24b3c472e914c671f8fbba518`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:05.321138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:05.438521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.440097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 970/1954] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:05.522869 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99931ba0-d988-47b0-9565-d4aae0cf3603 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.528822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.529756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.530048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.530329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.531045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.533066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.561744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 985/1955] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:05 2022] POST /identity/v3/projects => generated 407 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:05.565938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.577271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83bef5-72eb-439f-86c8-fca331cd103d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.636233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:05.648867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.650045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.650498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.651079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.651720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.654693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.688438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 971/1956] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:05 2022] POST /identity/v3/users => generated 310 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:05.689877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.703146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07bcf8-cc8c-4f98-9582-8447d228bfb6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.710905 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.712593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.713031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.713450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.714539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.716322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.729573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 986/1957] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:05.733673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.747300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a459086-a618-40dd-9722-13c80784faa4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.752719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.753567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.753851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.754129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] PATH_INFO: `/v3/projects/5b9c652297654d179650466fb5275637/users/6b45746112f7486482eb2ebdb8429778/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.773317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9c652297654d179650466fb5275637, user_id=6b45746112f7486482eb2ebdb8429778, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.776456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.795070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 972/1958] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:05 2022] PUT /identity/v3/projects/5b9c652297654d179650466fb5275637/users/6b45746112f7486482eb2ebdb8429778/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:05.799484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.815706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd935b9-3353-4f35-88ca-15b05f825aae admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.906201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be1e65-7e64-470b-9bf2-1745e4aa5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.907053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be1e65-7e64-470b-9bf2-1745e4aa5473 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.907407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be1e65-7e64-470b-9bf2-1745e4aa5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.907690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be1e65-7e64-470b-9bf2-1745e4aa5473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.908375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be1e65-7e64-470b-9bf2-1745e4aa5473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.912068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be1e65-7e64-470b-9bf2-1745e4aa5473 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.921989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 987/1959] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:05.926259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:05.953076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fcff1-a3a3-485f-b643-c667a82d999b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:05.958621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:05.959515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.959895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.960378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] PATH_INFO: `/v3/projects/5b9c652297654d179650466fb5275637/users/6b45746112f7486482eb2ebdb8429778/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:05.962605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9c652297654d179650466fb5275637, user_id=6b45746112f7486482eb2ebdb8429778, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:05.965518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:05.978194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 973/1960] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:05 2022] PUT /identity/v3/projects/5b9c652297654d179650466fb5275637/users/6b45746112f7486482eb2ebdb8429778/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:05.993894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:05.994388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:05.994779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:06.016690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:06.034336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] MFA Rules not processed for user `6b45746112f7486482eb2ebdb8429778`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:06.035671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:06.124644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 988/1961] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:05 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:06.135705 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:06.137533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:06.198791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeebb18-2b62-4063-a33c-9cf594ec596c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:06.202420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:06.203023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:06.203299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:06.203566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:06.212440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:06.213858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:06.223045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 974/1962] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:06 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:06.384758 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:06.394091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ffbed-bc38-41d4-b64b-aa7bc3c80f84 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:06.398240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffbef1-8681-4c84-8096-0e2e424b1bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:06.398841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffbef1-8681-4c84-8096-0e2e424b1bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:06.399130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffbef1-8681-4c84-8096-0e2e424b1bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:06.399409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffbef1-8681-4c84-8096-0e2e424b1bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:06.458103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffbef1-8681-4c84-8096-0e2e424b1bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:06.459696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffbef1-8681-4c84-8096-0e2e424b1bd8 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:06.470229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 989/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:06 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:08.366808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:08.368333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:08.465517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5c9a5-0104-4c04-977a-684b06a10f1e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:08.471871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:08.473844 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:08.474491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:08.475168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] PATH_INFO: `/v3/users/eaad4077d09f4ee8bd3460224e1b443f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:08.476664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaad4077d09f4ee8bd3460224e1b443f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:08.478907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:08.526979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaad4077d09f4ee8bd3460224e1b443f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:08.528738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:08.537813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaad4077d09f4ee8bd3460224e1b443f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:08.558781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaad4077d09f4ee8bd3460224e1b443f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:08.559814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaad4077d09f4ee8bd3460224e1b443f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:08.569210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaad4077d09f4ee8bd3460224e1b443f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:08.580621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 975/1964] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:08 2022] DELETE /identity/v3/users/eaad4077d09f4ee8bd3460224e1b443f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:08.587169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:08.587694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:08.588022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:08.618353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:08.620201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:08.686796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 990/1965] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:08.696723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:08.698236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:08.759915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf858f1-8099-4ce9-b98a-04b94317989c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:08.764155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:08.764808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:08.765162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:08.765497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:08.777175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:08.779610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:08.791313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 976/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:09.060847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:09.062325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:09.084920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eb226-258d-47e2-a07e-9d8e9e0e4810 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:09.091179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:09.091776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:09.092080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:09.092363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] PATH_INFO: `/v3/projects/3d25b0fef6844b24bbee39714b22c228` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:09.093505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d25b0fef6844b24bbee39714b22c228)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:09.095409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:09.128320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:09.149016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d25b0fef6844b24bbee39714b22c228', 'request_id': 'req-1ba205dc-d974-4e53-ad63-d947b596aa27'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.155826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d25b0fef6844b24bbee39714b22c228', 'request_id': 'req-1ba205dc-d974-4e53-ad63-d947b596aa27'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.167054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d25b0fef6844b24bbee39714b22c228', 'request_id': 'req-1ba205dc-d974-4e53-ad63-d947b596aa27'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.175150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] The token cache is being invalidate because project 3d25b0fef6844b24bbee39714b22c228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:09.175447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.176142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.177982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 991/1967] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:09 2022] DELETE /identity/v3/projects/3d25b0fef6844b24bbee39714b22c228 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:09.181939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:09.192592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac4083-43a8-4719-b783-65c49f35663c service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:09.256275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:09.256977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:09.257198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:09.257485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] PATH_INFO: `/v3/users/95e3ff1928fd441a95c3a21dd9ec12d9` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:09.258605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e3ff1928fd441a95c3a21dd9ec12d9)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:09.260607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:09.319880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e3ff1928fd441a95c3a21dd9ec12d9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.328247 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e3ff1928fd441a95c3a21dd9ec12d9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.350981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:09.352945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e3ff1928fd441a95c3a21dd9ec12d9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.353849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e3ff1928fd441a95c3a21dd9ec12d9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.364582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bca407ac-80a9-4854-baff-fceb487b296c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e3ff1928fd441a95c3a21dd9ec12d9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:09.375995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 977/1968] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:09 2022] DELETE /identity/v3/users/95e3ff1928fd441a95c3a21dd9ec12d9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:09.917908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:09.919517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:09.933289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba205dc-d974-4e53-ad63-d947b596aa27 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:09.991002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:09.991635 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:09.991918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:09.992233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] PATH_INFO: `/v3/projects/1284bf1ffb05415f80079d778b4b66f0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:09.993362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1284bf1ffb05415f80079d778b4b66f0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:09.995262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.044827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1284bf1ffb05415f80079d778b4b66f0', 'request_id': 'req-cc0e88a7-6559-4de6-8598-b5c1abb8632e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:10.050892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1284bf1ffb05415f80079d778b4b66f0', 'request_id': 'req-cc0e88a7-6559-4de6-8598-b5c1abb8632e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:10.059917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1284bf1ffb05415f80079d778b4b66f0', 'request_id': 'req-cc0e88a7-6559-4de6-8598-b5c1abb8632e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:10.070404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] The token cache is being invalidate because project 1284bf1ffb05415f80079d778b4b66f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:10.070689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:10.071308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc0e88a7-6559-4de6-8598-b5c1abb8632e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:10.072837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 992/1969] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:09 2022] DELETE /identity/v3/projects/1284bf1ffb05415f80079d778b4b66f0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:10.077673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.077985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.078258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.099581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:10.100729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:10.159183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 978/1970] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:10.166885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.167170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.167473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.187244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:10.188473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:10.198163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 993/1971] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:10.202514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.212483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21936c-b195-41f0-9ece-e60a1e26dfe4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.216946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.217567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.217842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.218111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.218748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:10.219048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.220672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.231979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 979/1972] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:10.236306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.246603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dbc98-63a2-4608-89a8-668edd4433c0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.252555 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.253312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.253708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.254060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.255839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.261479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.285435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 994/1973] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:10.289835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.300115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bf32b-90d5-4a13-8c9f-39ecd647c6c8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.355999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.356604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.356883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.357160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.357792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.359761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.381237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 980/1974] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:10.385066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.395326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7a5e2-0a5e-483f-bdd8-40dc63ae1363 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.400134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.400735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.401054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.401366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.402013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.403580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.414269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 995/1975] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:10.417960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.428236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609e17a-60b5-4df3-bca3-3fdf71dbf84c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.432715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec462e-2b6b-471b-b6c8-d50a06633aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.433309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec462e-2b6b-471b-b6c8-d50a06633aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.433585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec462e-2b6b-471b-b6c8-d50a06633aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.433851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec462e-2b6b-471b-b6c8-d50a06633aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.434483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec462e-2b6b-471b-b6c8-d50a06633aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.436097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec462e-2b6b-471b-b6c8-d50a06633aaa admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.447416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 981/1976] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:10.451781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.458819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:10.462842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a4807-4c3a-4de7-8d6e-ee420b0b5404 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.469909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.470612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.470884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.471163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] PATH_INFO: `/v3/projects/37814d3c8e46464dad5418c1b423ee37/users/962771ab2e354dcd99b56b3a6bba0723/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.493645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37814d3c8e46464dad5418c1b423ee37, user_id=962771ab2e354dcd99b56b3a6bba0723, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.496472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.509840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 996/1977] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:10 2022] PUT /identity/v3/projects/37814d3c8e46464dad5418c1b423ee37/users/962771ab2e354dcd99b56b3a6bba0723/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:10.519972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.520478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.520938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.548782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] MFA Rules not processed for user `962771ab2e354dcd99b56b3a6bba0723`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:10.550044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:10.593403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:10.618173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 982/1978] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:10.622604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.685982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed2671-0a8f-495c-b817-1641aa4987c6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.690600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.691206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.691491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.691810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.692557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.694458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.725202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 997/1979] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/projects => generated 397 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:10.729172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.741510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f3eef-b071-4296-8e46-0f787faeb8e5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.808428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.809082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.809374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.809640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.810287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.812262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.839109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 983/1980] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:10 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:10.840388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.851410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737704c-a4d8-4d0d-b9f7-0ec39022c309 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.860611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.861684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.861910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.862145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.862986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.865785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.877860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 998/1981] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:10.886219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.893059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d84fb9-ed5b-4b3a-b66a-892ee2a9bacd admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:10.898219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:10.898882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:10.899165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:10.899438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] PATH_INFO: `/v3/projects/6b2cea5170df43c2915abfe6dd89d8e5/users/03eac40ecee748c792ba0ecef7dd86f5/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:10.917358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2cea5170df43c2915abfe6dd89d8e5, user_id=03eac40ecee748c792ba0ecef7dd86f5, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:10.920092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:10.942543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 984/1982] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:10 2022] PUT /identity/v3/projects/6b2cea5170df43c2915abfe6dd89d8e5/users/03eac40ecee748c792ba0ecef7dd86f5/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:10.947868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:10.963783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7512f-bb4e-42b1-9522-bda015724fe9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:11.040611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d267fd-61b6-4045-8f6f-2eac12cd124d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:11.041231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d267fd-61b6-4045-8f6f-2eac12cd124d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:11.041516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d267fd-61b6-4045-8f6f-2eac12cd124d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:11.041794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d267fd-61b6-4045-8f6f-2eac12cd124d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:11.042447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d267fd-61b6-4045-8f6f-2eac12cd124d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:11.044238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d267fd-61b6-4045-8f6f-2eac12cd124d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:11.058176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 999/1983] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:11.059375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:11.070964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513b3a-57f1-4ce7-bb27-4be31eee5eaf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:11.076045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:11.076715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:11.076999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:11.077276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] PATH_INFO: `/v3/projects/6b2cea5170df43c2915abfe6dd89d8e5/users/03eac40ecee748c792ba0ecef7dd86f5/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:11.079208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2cea5170df43c2915abfe6dd89d8e5, user_id=03eac40ecee748c792ba0ecef7dd86f5, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:11.082004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:11.092871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 985/1984] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:11 2022] PUT /identity/v3/projects/6b2cea5170df43c2915abfe6dd89d8e5/users/03eac40ecee748c792ba0ecef7dd86f5/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:11.104001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:11.104362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:11.104708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:11.134906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] MFA Rules not processed for user `03eac40ecee748c792ba0ecef7dd86f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:11.136034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:11.175731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:11.205268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1000/1985] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:11 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:11.216001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:11.217672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:11.276853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80980ec-dd1c-4e83-97b3-e8644aba50e7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:11.281033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:11.281534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:11.281887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:11.282080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:11.292918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:11.294350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:11.306652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 986/1986] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:11.389531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:11.400616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4b41c-ba81-4544-aa8f-516c3e85be8d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:11.404515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc39053-4074-48be-a529-f60aeb3be18e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:11.405231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc39053-4074-48be-a529-f60aeb3be18e service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:11.405519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc39053-4074-48be-a529-f60aeb3be18e service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:11.405797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc39053-4074-48be-a529-f60aeb3be18e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:11.463141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc39053-4074-48be-a529-f60aeb3be18e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:11.464622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc39053-4074-48be-a529-f60aeb3be18e service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:11.475325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1001/1987] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:20.391845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:20.394178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:20.468087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:20.507635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831325b-965c-4914-890e-91a36bda27fd service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:20.518478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:20.519478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:20.519989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:20.520516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] PATH_INFO: `/v3/users/ef59e4477467484080a04bfc8cc67387` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:20.523485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef59e4477467484080a04bfc8cc67387)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:20.527095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:20.610054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef59e4477467484080a04bfc8cc67387'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.623196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef59e4477467484080a04bfc8cc67387'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.648759 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef59e4477467484080a04bfc8cc67387'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.650480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef59e4477467484080a04bfc8cc67387'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.661535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef59e4477467484080a04bfc8cc67387'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.671465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 987/1988] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:20 2022] DELETE /identity/v3/users/ef59e4477467484080a04bfc8cc67387 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:20.676294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:20.677815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:20.678280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:20.702942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:20.705168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:20.714914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:20.716679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:20.786205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1002/1989] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:20.786867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:20.788333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:20.800966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03b4a0-7183-404e-acdd-6adfb1e73fd2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:20.806378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:20.807094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:20.807476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:20.807828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] PATH_INFO: `/v3/users/252f69423e3045d28a73f952c21f8116` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:20.809057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=252f69423e3045d28a73f952c21f8116)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:20.810996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:20.812330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0e75d-a5d1-43d0-9db7-e49170b95c12 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:20.818727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:20.819570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:20.820020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:20.820425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] PATH_INFO: `/v3/users/6803aea24b3c472e914c671f8fbba518` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:20.821843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6803aea24b3c472e914c671f8fbba518)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:20.824672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:20.869558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252f69423e3045d28a73f952c21f8116'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.878559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252f69423e3045d28a73f952c21f8116'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.880946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252f69423e3045d28a73f952c21f8116'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.882762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6803aea24b3c472e914c671f8fbba518'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.890101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252f69423e3045d28a73f952c21f8116'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.893583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6803aea24b3c472e914c671f8fbba518'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.897414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:20.902414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:20.907458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252f69423e3045d28a73f952c21f8116'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.914637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1003/1990] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:20 2022] DELETE /identity/v3/users/252f69423e3045d28a73f952c21f8116 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:20.917775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:20.917775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:20.917954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6803aea24b3c472e914c671f8fbba518'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.918797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6803aea24b3c472e914c671f8fbba518'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.927253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-aa5bb55e-813b-4ed3-a395-bbd6d4108068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6803aea24b3c472e914c671f8fbba518'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:20.933536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 988/1991] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:20 2022] DELETE /identity/v3/users/6803aea24b3c472e914c671f8fbba518 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:20.934761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c02ec-b6d8-43d7-ac8c-d4b2b00c44d3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:20.935043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c02ec-b6d8-43d7-ac8c-d4b2b00c44d3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:20.935315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c02ec-b6d8-43d7-ac8c-d4b2b00c44d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:20.958440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5f7c02ec-b6d8-43d7-ac8c-d4b2b00c44d3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:20.959576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5f7c02ec-b6d8-43d7-ac8c-d4b2b00c44d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:20.983665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab034a-e3c4-4e0e-bb4e-946d9fa0e7a8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:21.048618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 989/1992] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:21.050298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.050756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.051858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.075058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:21.076345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.085637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 990/1993] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:21.089841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.091278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.102774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:21.103575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.104084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.104360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.107355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.128297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:21.130412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:21.150250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1004/1994] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:21.151470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.153775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.170722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe9db-0b13-4b8a-bed7-4690786368dc service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:21.174763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8a8a3-d023-4996-803f-7f9387d956bc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:21.177579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:21.178175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.178458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.178734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.179766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:21.180405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.180788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.181382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.193879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:21.194592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:21.195502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:21.196265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:21.209263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1005/1995] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:21.217058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 991/1996] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:21.671760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.674059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.691522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3ef97-7d27-478f-93c0-2716b3b49eec service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:21.699480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:21.700758 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.701331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.701653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] PATH_INFO: `/v3/projects/a0703401286f4528948eb901934de0c4` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.703771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0703401286f4528948eb901934de0c4)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:21.707209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:21.775546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0703401286f4528948eb901934de0c4', 'request_id': 'req-568abd7d-c2bc-4556-9129-694c603952d3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.780386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.781812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.791483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0703401286f4528948eb901934de0c4', 'request_id': 'req-568abd7d-c2bc-4556-9129-694c603952d3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.798984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18b98b-2ff0-4e6e-8b4c-277d9082f58f service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:21.803071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0703401286f4528948eb901934de0c4', 'request_id': 'req-568abd7d-c2bc-4556-9129-694c603952d3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.823223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] The token cache is being invalidate because project a0703401286f4528948eb901934de0c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:21.823518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.824651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.826398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1006/1997] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:21 2022] DELETE /identity/v3/projects/a0703401286f4528948eb901934de0c4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:21.827184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.828744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.849558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:21.870625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:21.871257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.871532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.871800 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] PATH_INFO: `/v3/projects/02df7e01cd074fc88ecbe4a054500fea` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.873105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02df7e01cd074fc88ecbe4a054500fea)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:21.874994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:21.892789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:21.901961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abd7d-c2bc-4556-9129-694c603952d3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:21.907229 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:21.907939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:21.908320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:21.908671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] PATH_INFO: `/v3/projects/37a895bf3cac4122bc1c7ead37cb152c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:21.909879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a895bf3cac4122bc1c7ead37cb152c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:21.912391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:21.925696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02df7e01cd074fc88ecbe4a054500fea', 'request_id': 'req-8619c488-710b-4598-8d26-445cb346d087'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.932110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02df7e01cd074fc88ecbe4a054500fea', 'request_id': 'req-8619c488-710b-4598-8d26-445cb346d087'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.938464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02df7e01cd074fc88ecbe4a054500fea', 'request_id': 'req-8619c488-710b-4598-8d26-445cb346d087'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.952271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] The token cache is being invalidate because project 02df7e01cd074fc88ecbe4a054500fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:21.952690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.953495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.955323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 992/1998] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:21 2022] DELETE /identity/v3/projects/02df7e01cd074fc88ecbe4a054500fea => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:21.956024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.957920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:21.965178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a895bf3cac4122bc1c7ead37cb152c', 'request_id': 'req-71a2d825-565c-41f2-99f1-b7f64addf87e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.971432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a895bf3cac4122bc1c7ead37cb152c', 'request_id': 'req-71a2d825-565c-41f2-99f1-b7f64addf87e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.980030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a895bf3cac4122bc1c7ead37cb152c', 'request_id': 'req-71a2d825-565c-41f2-99f1-b7f64addf87e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.991166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] The token cache is being invalidate because project 37a895bf3cac4122bc1c7ead37cb152c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:21.991554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.992259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:21.993923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1007/1999] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:21 2022] DELETE /identity/v3/projects/37a895bf3cac4122bc1c7ead37cb152c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:21.994700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:21.998258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:22.028259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c488-710b-4598-8d26-445cb346d087 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:22.069491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2d825-565c-41f2-99f1-b7f64addf87e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:22.074233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:22.074865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.075149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.075428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] PATH_INFO: `/v3/users/436128f89a0849eebaa1a720527d9b0e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.076624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436128f89a0849eebaa1a720527d9b0e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:22.078561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:22.092368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:22.093268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.093773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.094162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] PATH_INFO: `/v3/users/9f517c0c28504ef5a24116c53b734080` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.095381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f517c0c28504ef5a24116c53b734080)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:22.097402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:22.144986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436128f89a0849eebaa1a720527d9b0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.146203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:22.149243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f517c0c28504ef5a24116c53b734080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.153307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:22.158818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f517c0c28504ef5a24116c53b734080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.168558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436128f89a0849eebaa1a720527d9b0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.169310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436128f89a0849eebaa1a720527d9b0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.175375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f517c0c28504ef5a24116c53b734080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.176782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f517c0c28504ef5a24116c53b734080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.179058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436128f89a0849eebaa1a720527d9b0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.186429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f517c0c28504ef5a24116c53b734080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.196419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 993/2000] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:21 2022] DELETE /identity/v3/users/9f517c0c28504ef5a24116c53b734080 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:22.196419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:22.196419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:22.202216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436128f89a0849eebaa1a720527d9b0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.208740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1008/2001] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:21 2022] DELETE /identity/v3/users/436128f89a0849eebaa1a720527d9b0e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:22.297976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d1157-a3fb-4582-9f15-c8522f7e1a34 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:22.377238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:22.378210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.378210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.378436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] PATH_INFO: `/v3/users/6b45746112f7486482eb2ebdb8429778` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.380655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b45746112f7486482eb2ebdb8429778)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:22.382744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:22.448988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b45746112f7486482eb2ebdb8429778'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.460035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b45746112f7486482eb2ebdb8429778'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.480935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b45746112f7486482eb2ebdb8429778'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.481815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b45746112f7486482eb2ebdb8429778'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.490406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:22.492965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b45746112f7486482eb2ebdb8429778'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.508772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 994/2002] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:22 2022] DELETE /identity/v3/users/6b45746112f7486482eb2ebdb8429778 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:22.636906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:22.638602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:22.654148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b2bb-baae-4347-b4c8-86f984894ed3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:22.662068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:22.663545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:22.782900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dddf7-c697-40ba-a69d-b03dac8091b5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:22.783808 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:22.784495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.784777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.785068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] PATH_INFO: `/v3/projects/1e1b4fe9aa794255a742bc9570766367` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.787175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1b4fe9aa794255a742bc9570766367)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:22.789627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:22.790744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:22.792582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.793055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.793498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] PATH_INFO: `/v3/projects/896a918fed814461ad6217fb043f7773` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.795631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896a918fed814461ad6217fb043f7773)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:22.797792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:22.852802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1b4fe9aa794255a742bc9570766367', 'request_id': 'req-3bd15fdf-a694-4a09-8292-1ed3201ce519'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.856280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896a918fed814461ad6217fb043f7773', 'request_id': 'req-40b511d9-0012-44bb-8dd0-f67121cebc74'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.861335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1b4fe9aa794255a742bc9570766367', 'request_id': 'req-3bd15fdf-a694-4a09-8292-1ed3201ce519'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.863237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896a918fed814461ad6217fb043f7773', 'request_id': 'req-40b511d9-0012-44bb-8dd0-f67121cebc74'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.870079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896a918fed814461ad6217fb043f7773', 'request_id': 'req-40b511d9-0012-44bb-8dd0-f67121cebc74'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.877448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1b4fe9aa794255a742bc9570766367', 'request_id': 'req-3bd15fdf-a694-4a09-8292-1ed3201ce519'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.888580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] The token cache is being invalidate because project 896a918fed814461ad6217fb043f7773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:22.888999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.889724 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-40b511d9-0012-44bb-8dd0-f67121cebc74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.891550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 995/2003] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:22 2022] DELETE /identity/v3/projects/896a918fed814461ad6217fb043f7773 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:22.893097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] The token cache is being invalidate because project 1e1b4fe9aa794255a742bc9570766367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:22.893455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.894308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3bd15fdf-a694-4a09-8292-1ed3201ce519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:22.896647 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.897005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.897528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1009/2004] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:22 2022] DELETE /identity/v3/projects/1e1b4fe9aa794255a742bc9570766367 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:22.897636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.905389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfce9-7f60-4c2c-b0eb-af5f60683a56 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:22.905745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfce9-7f60-4c2c-b0eb-af5f60683a56 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:22.906070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bfce9-7f60-4c2c-b0eb-af5f60683a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:22.925831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:22.927455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:22.943680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7e6bfce9-7f60-4c2c-b0eb-af5f60683a56 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:22.945513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7e6bfce9-7f60-4c2c-b0eb-af5f60683a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.038147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 996/2005] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.039348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.041644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.068570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938a681-9f61-43f8-adfe-d30194c2426c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.082750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:23.093477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.097492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.098919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.100308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] PATH_INFO: `/v3/projects/5b9c652297654d179650466fb5275637` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.103776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b9c652297654d179650466fb5275637)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.110402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1010/2006] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.110772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.113109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ed39a-e70c-4e04-94bb-1bd1459ee90c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.115591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ed39a-e70c-4e04-94bb-1bd1459ee90c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.116050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ed39a-e70c-4e04-94bb-1bd1459ee90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.143450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-337ed39a-e70c-4e04-94bb-1bd1459ee90c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:23.144864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-337ed39a-e70c-4e04-94bb-1bd1459ee90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.154810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1011/2007] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.156044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1efaf-bf75-4c6e-8a45-ec01a9b0be99 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.156352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:23.156488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1efaf-bf75-4c6e-8a45-ec01a9b0be99 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.156716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1efaf-bf75-4c6e-8a45-ec01a9b0be99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.180225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a7e1efaf-bf75-4c6e-8a45-ec01a9b0be99 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:23.182320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a7e1efaf-bf75-4c6e-8a45-ec01a9b0be99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.192757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b9c652297654d179650466fb5275637', 'request_id': 'req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.199559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b9c652297654d179650466fb5275637', 'request_id': 'req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.205355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b9c652297654d179650466fb5275637', 'request_id': 'req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.220359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] The token cache is being invalidate because project 5b9c652297654d179650466fb5275637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:23.220847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.221907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.224305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 997/2008] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:23 2022] DELETE /identity/v3/projects/5b9c652297654d179650466fb5275637 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:23.224987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.226279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.262863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1012/2009] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.264188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.265031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.265031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.292160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:23.293898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.302942 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1013/2010] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.303811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.313561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566be4e5-0525-49d0-9042-5fc91c913da5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.319273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.319884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.320204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.320481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.321152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:23.321441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.322378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa390a-fc76-4186-88bb-9a89df6a3bf6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.323154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2d43e-c3ec-437d-ba7a-43f6853d8ac6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.328590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.329232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.329516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.329787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.330439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:23.330737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.335391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1014/2011] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:23.335697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.336650 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.336945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.337333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.351047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 998/2012] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:23.352199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.359120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:23.360362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.365066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b49201-0769-4625-a924-524c88a60afc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.369402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1015/2013] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.372960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.373068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.373068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.373068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.373068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.373467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.374518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.384214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7c7c7-5b17-4800-8ec9-b286e762957d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.397477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.398451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.399137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.399477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.400355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.402539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.412470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 999/2014] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:23.413465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.436111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1016/2015] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:23.437137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.439091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.485334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea04b5-7e21-4ed3-a980-d40c470632b5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.506267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7ddf-0082-440b-93b5-15b7c6e10cf0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.513019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.513773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.514059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.514335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] PATH_INFO: `/v3/users/962771ab2e354dcd99b56b3a6bba0723` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.515478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:23.517791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] RBAC: Authorizing `identity:delete_user(user_id=962771ab2e354dcd99b56b3a6bba0723)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.520971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.560380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.561062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.561325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.561642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.569452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:23.569452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.569452 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.601778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:23.603793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1000/2016] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:23.604806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.607672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '962771ab2e354dcd99b56b3a6bba0723'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.620101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '962771ab2e354dcd99b56b3a6bba0723'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.621338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '962771ab2e354dcd99b56b3a6bba0723'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.632587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa67927-3e15-4bc7-b344-0531242252d9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.635129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '962771ab2e354dcd99b56b3a6bba0723'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.643228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.644982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.645453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.646279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.652141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.654800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '962771ab2e354dcd99b56b3a6bba0723'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:23.655099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.664264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1017/2017] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:23 2022] DELETE /identity/v3/users/962771ab2e354dcd99b56b3a6bba0723 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:23.665148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.668274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.685819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1001/2018] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/users => generated 287 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:23.686696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.688422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.720495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ede457-0e7b-45c2-8983-70640793c7c4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.761789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26199cd4-1cc2-47fc-9df9-53979092d8da admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.767399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253329-4f0d-4403-bcc3-4f11a1fae1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.768484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253329-4f0d-4403-bcc3-4f11a1fae1eb admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.768484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253329-4f0d-4403-bcc3-4f11a1fae1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.768720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253329-4f0d-4403-bcc3-4f11a1fae1eb admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.769308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253329-4f0d-4403-bcc3-4f11a1fae1eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.775283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253329-4f0d-4403-bcc3-4f11a1fae1eb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.802997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1018/2019] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:23.813675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.814069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.814472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.815013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.815630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.815907 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.816254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.817030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.819779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.839859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:23.841125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.853747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1019/2020] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:23.854580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.857582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.871100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1002/2021] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:23 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:23.871100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.871338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c170ea-d109-4497-a62c-8c5732c98eea None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.942812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.943560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.943852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.944164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.944823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.946422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:23.958517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1020/2022] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:23.959412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:23.961251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:23.970795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5694-9abb-4e7e-a806-f82eb1c89c49 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:23.979040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:23.979687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:23.980110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:23.980401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:23.981109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:23.983662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.009487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1003/2023] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:24.010341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.016541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:24.022870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d2089-a82e-4f2e-bbed-803adefef245 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.034445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.037607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.037894 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.038170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.038813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.039906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3befb-8112-4e80-9834-2d17fa173967 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.044464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.049782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.050621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.050838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.052121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.089339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.090961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.105321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1021/2024] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:24.106338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.116206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1004/2025] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:24.118765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.122967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b0e11-9375-4d9b-b0c6-a72dcec513a7 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.129880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.131270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.131270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.132057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.132978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.134344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.147344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af755a-feff-4aa8-bf44-34c14ab78d75 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.149440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1022/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:24.150703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.153499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.154370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.154652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.154933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.155730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.157403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.162731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:24.167054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de208-2594-4c4a-ac9a-c1f3b78738ef admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.173146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.174757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.175339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.175796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.177389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1005/2027] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:24.178561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.181137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.181842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.199312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e90700-29bf-4279-a6fc-95a816d4f738 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.200334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1023/2028] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:24.201626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.205088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4be67-03d1-455b-a8c0-457e97aa3a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.205897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4be67-03d1-455b-a8c0-457e97aa3a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.206249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4be67-03d1-455b-a8c0-457e97aa3a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.206594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4be67-03d1-455b-a8c0-457e97aa3a57 admin admin] PATH_INFO: `/v3/projects/a7089141e0b44798b98b58d9673176c4/users/0dafe42edecc4d6a83d94e3805263e51/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.215230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64ef03-4a4d-42ba-9a8d-8c87e7ed26c7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.221077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.222209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.222669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.223658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] PATH_INFO: `/v3/projects/beb67fcf6f8f4b16964737fc623508cc/users/32b5440737524485bcb815cf05949995/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.245019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4be67-03d1-455b-a8c0-457e97aa3a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7089141e0b44798b98b58d9673176c4, user_id=0dafe42edecc4d6a83d94e3805263e51, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.247873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb67fcf6f8f4b16964737fc623508cc, user_id=32b5440737524485bcb815cf05949995, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.249410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4be67-03d1-455b-a8c0-457e97aa3a57 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.251874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.273527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1024/2030] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:24 2022] PUT /identity/v3/projects/beb67fcf6f8f4b16964737fc623508cc/users/32b5440737524485bcb815cf05949995/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:24.276018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.287280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1006/2030] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:24 2022] PUT /identity/v3/projects/a7089141e0b44798b98b58d9673176c4/users/0dafe42edecc4d6a83d94e3805263e51/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:24.291444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.292184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.293200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.338413 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] MFA Rules not processed for user `32b5440737524485bcb815cf05949995`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:24.340904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:24.378433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91529acb-1318-4cd0-b7e5-96e7ccc82f17 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.384406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6eb75-43ba-4da7-808d-9ce1634dd044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.385044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6eb75-43ba-4da7-808d-9ce1634dd044 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.385346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6eb75-43ba-4da7-808d-9ce1634dd044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.385651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6eb75-43ba-4da7-808d-9ce1634dd044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.386400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6eb75-43ba-4da7-808d-9ce1634dd044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.388160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6eb75-43ba-4da7-808d-9ce1634dd044 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.402785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1025/2031] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:24.404870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bb2a6-3168-41d1-8b6f-776cfd76e938 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.405336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bb2a6-3168-41d1-8b6f-776cfd76e938 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.405753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bb2a6-3168-41d1-8b6f-776cfd76e938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.429212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1007/2032] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:24.434911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.449291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c9d22-30ad-4167-8b0f-1e0a3d9fbfdc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.452708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-100bb2a6-3168-41d1-8b6f-776cfd76e938 None None] MFA Rules not processed for user `0dafe42edecc4d6a83d94e3805263e51`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:24.454450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-100bb2a6-3168-41d1-8b6f-776cfd76e938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:24.455411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.456118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.456434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.457393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] PATH_INFO: `/v3/projects/e91956bfef8d47c7a47b71e39d8160ce/users/2319cf8b699d4ffb8dc907dade820b20/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.475975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91956bfef8d47c7a47b71e39d8160ce, user_id=2319cf8b699d4ffb8dc907dade820b20, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.479300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.498371 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1008/2033] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:24 2022] PUT /identity/v3/projects/e91956bfef8d47c7a47b71e39d8160ce/users/2319cf8b699d4ffb8dc907dade820b20/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:24.500251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.555968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1026/2034] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:24.557900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34720-2974-4616-999f-e67a37b71d9a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.558330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34720-2974-4616-999f-e67a37b71d9a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.558755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34720-2974-4616-999f-e67a37b71d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.593346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-add34720-2974-4616-999f-e67a37b71d9a None None] MFA Rules not processed for user `2319cf8b699d4ffb8dc907dade820b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:24.594843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-add34720-2974-4616-999f-e67a37b71d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:24.606713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:24.617382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba7aa-3a41-433a-a3cb-f32183587622 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.623519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.624498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.625498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.626120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.626930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.630996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.666487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1009/2035] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/projects => generated 363 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:24.667846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.669727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:24.684587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1027/2036] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:24.685972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add34720-2974-4616-999f-e67a37b71d9a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.701787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add34720-2974-4616-999f-e67a37b71d9a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.706989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:24.753083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e3964-254f-4556-9a4f-cabea464baf6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.757073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.757680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.757978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.758246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.768982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.769717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.770081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.770420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.771124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.773169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.815900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1028/2037] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:24.815900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.835695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67719f63-3162-4fcd-ac09-c892b03ffef0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:24.849936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:24.850725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:24.851015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:24.851295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:24.854275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.858956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.901847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:24.906690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:24.920537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1029/2038] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:24 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:24.920537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.922651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:24.928166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1010/2039] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:24.930586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:24.950236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4bfd-fa77-4a8a-9410-9029b3633f6f service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.055292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eeffa6-754e-485e-8ab3-2539ab07de47 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.060947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.061792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.062150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.062663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] PATH_INFO: `/v3/projects/37814d3c8e46464dad5418c1b423ee37` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.063277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.064124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.064240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=37814d3c8e46464dad5418c1b423ee37)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.064762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.065381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.066375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.067763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.070690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.086272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1011/2040] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:25.087938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.102259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:25.104248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78640-fd1e-43cf-9b18-c41cb59b8b67 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.124917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.125661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.126051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.126344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.127331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.130626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.167450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37814d3c8e46464dad5418c1b423ee37', 'request_id': 'req-b994f63a-7ca8-4409-8230-9b245e028baa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.174762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1012/2041] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:25 2022] POST /identity/v3/users => generated 290 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:25.175722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.177709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37814d3c8e46464dad5418c1b423ee37', 'request_id': 'req-b994f63a-7ca8-4409-8230-9b245e028baa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.189179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37814d3c8e46464dad5418c1b423ee37', 'request_id': 'req-b994f63a-7ca8-4409-8230-9b245e028baa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.199039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cebbb-70a4-4374-a95a-3e3b425db00e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.218103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] The token cache is being invalidate because project 37814d3c8e46464dad5418c1b423ee37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:25.218333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.219025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.221082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1030/2042] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:24 2022] DELETE /identity/v3/projects/37814d3c8e46464dad5418c1b423ee37 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:25.222621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.226169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.297521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:25.308257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.309038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.309318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.309592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] PATH_INFO: `/v3/projects/bccb1f5178c6486bbc48d58e703e48df/users/d5ae9dd8de934c64ad5a85aebae3505a/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.337241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccb1f5178c6486bbc48d58e703e48df, user_id=d5ae9dd8de934c64ad5a85aebae3505a, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.342536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.349308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f63a-7ca8-4409-8230-9b245e028baa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.354644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.355358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.356087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.356087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.356726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.358679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.364245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1013/2043] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:25 2022] PUT /identity/v3/projects/bccb1f5178c6486bbc48d58e703e48df/users/d5ae9dd8de934c64ad5a85aebae3505a/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:25.367878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.369584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.373622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1031/2044] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:25.373622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.390041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.437919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a36550-5aab-4fcf-a06e-147ca17e37ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.511490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb5414-604b-41c5-a36f-418b00674aad admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.519139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.521663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.522204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.522685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] PATH_INFO: `/v3/users/03eac40ecee748c792ba0ecef7dd86f5` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.524781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03eac40ecee748c792ba0ecef7dd86f5)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.527773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.559484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.560148 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.560431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.560703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.561450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.563080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.579082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1032/2045] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:25.579082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.592412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c877c9c-10c3-4635-aa7e-c340138bce42 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.598520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.599358 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.599782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.600135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.600741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.601433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03eac40ecee748c792ba0ecef7dd86f5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.602479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.611195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03eac40ecee748c792ba0ecef7dd86f5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.617979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1033/2046] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:25.617979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.635007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03eac40ecee748c792ba0ecef7dd86f5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.635007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03eac40ecee748c792ba0ecef7dd86f5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.635324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d45e1-e641-43f4-a2c0-d7ee8518b180 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.635324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00074acb-e1a1-4a06-bfe0-59d530e700f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.635324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00074acb-e1a1-4a06-bfe0-59d530e700f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.635324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00074acb-e1a1-4a06-bfe0-59d530e700f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.635668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00074acb-e1a1-4a06-bfe0-59d530e700f8 admin admin] PATH_INFO: `/v3/projects/bccb1f5178c6486bbc48d58e703e48df/users/d5ae9dd8de934c64ad5a85aebae3505a/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.636823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00074acb-e1a1-4a06-bfe0-59d530e700f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccb1f5178c6486bbc48d58e703e48df, user_id=d5ae9dd8de934c64ad5a85aebae3505a, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.639502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00074acb-e1a1-4a06-bfe0-59d530e700f8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.641836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03eac40ecee748c792ba0ecef7dd86f5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:25.648283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1014/2047] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:25 2022] DELETE /identity/v3/users/03eac40ecee748c792ba0ecef7dd86f5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:25.649048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.650447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.655989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1034/2048] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:25 2022] PUT /identity/v3/projects/bccb1f5178c6486bbc48d58e703e48df/users/d5ae9dd8de934c64ad5a85aebae3505a/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:25.665268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.665551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.665821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.697994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] MFA Rules not processed for user `d5ae9dd8de934c64ad5a85aebae3505a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:25.699276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.705331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:25.736064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370156-bbd9-4708-846e-6ac072a38272 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.743625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419f8bb-4977-4ed3-833d-586bb8c0390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.744325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419f8bb-4977-4ed3-833d-586bb8c0390d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.744606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419f8bb-4977-4ed3-833d-586bb8c0390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.744884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419f8bb-4977-4ed3-833d-586bb8c0390d admin admin] PATH_INFO: `/v3/projects/6fe2f942e1b54a349b448ba4f3d43d2d/users/dccbec7f1a0e48d480126eb4bc663220/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.748188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:25.763604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419f8bb-4977-4ed3-833d-586bb8c0390d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe2f942e1b54a349b448ba4f3d43d2d, user_id=dccbec7f1a0e48d480126eb4bc663220, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:25.766845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419f8bb-4977-4ed3-833d-586bb8c0390d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:25.784412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1015/2049] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:25 2022] PUT /identity/v3/projects/6fe2f942e1b54a349b448ba4f3d43d2d/users/dccbec7f1a0e48d480126eb4bc663220/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:25.793294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1035/2050] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:25.799007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.799347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.799560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:25.806347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.808122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.835425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] MFA Rules not processed for user `dccbec7f1a0e48d480126eb4bc663220`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:25.837268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:25.878642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39a636-82e5-4c44-8808-c55ff71846c5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:25.924884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1016/2051] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:25.929573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:25.945782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:25.946436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:25.946802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:25.947059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.016000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.016821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d4cbe-8c32-4a84-b42f-a723499562a0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.017608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.022257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.023074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.023506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.023850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.024447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.026667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.037568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1036/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:26.038500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.041143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.064043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf33e0-481a-42cc-a597-d78e3c6922b5 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.064043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.064043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.064306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.064567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] PATH_INFO: `/v3/projects/6b2cea5170df43c2915abfe6dd89d8e5` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.065793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2cea5170df43c2915abfe6dd89d8e5)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.068749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.072012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:26.080660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1017/2053] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:25 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:26.084625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.126030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2cea5170df43c2915abfe6dd89d8e5', 'request_id': 'req-ee8bde74-3866-4350-843b-8a1a25cb2efc'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.136156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2cea5170df43c2915abfe6dd89d8e5', 'request_id': 'req-ee8bde74-3866-4350-843b-8a1a25cb2efc'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.146793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2cea5170df43c2915abfe6dd89d8e5', 'request_id': 'req-ee8bde74-3866-4350-843b-8a1a25cb2efc'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.160135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] The token cache is being invalidate because project 6b2cea5170df43c2915abfe6dd89d8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:26.160646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.161416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ee8bde74-3866-4350-843b-8a1a25cb2efc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.163303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1037/2054] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:26 2022] DELETE /identity/v3/projects/6b2cea5170df43c2915abfe6dd89d8e5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:26.168037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:26.172304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e8a2-145f-420e-99ee-a80a6c42aa39 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.173671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72019-d81d-4790-81ab-8552fdb1de49 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.173986 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72019-d81d-4790-81ab-8552fdb1de49 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.174267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72019-d81d-4790-81ab-8552fdb1de49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.204854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-dca72019-d81d-4790-81ab-8552fdb1de49 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:26.206372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dca72019-d81d-4790-81ab-8552fdb1de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.245934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.246703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.247045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.247427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.248160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.250103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.273866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1018/2055] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:26.277590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.279629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.290697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1038/2056] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:26.294084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4e57-4c8f-4aaf-8440-b21df41ec4b5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.298816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.299397 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.299690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.302014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.302641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.302938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.303162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.303802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.305403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.318587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1019/2057] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:26.323575 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.326783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:26.328233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.337169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2658f-cc1c-4a02-9415-81fb3ab2aa8d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.340276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1039/2058] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:26.342109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.344818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.345510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.345789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.346060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] PATH_INFO: `/v3/projects/08153f05fd3d46b0bd8bfd67fef6de40/users/9eda944896e440009ced22ef6732acb7/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.352720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aabb9-418c-40ee-80b6-e8baf354aad7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.359810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.360531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.360955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.361144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.361787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:26.362092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.363307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08153f05fd3d46b0bd8bfd67fef6de40, user_id=9eda944896e440009ced22ef6732acb7, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.364570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.366896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.377040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1040/2059] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:40:26.381654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.387722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1020/2060] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:26 2022] PUT /identity/v3/projects/08153f05fd3d46b0bd8bfd67fef6de40/users/9eda944896e440009ced22ef6732acb7/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:26.394670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.394779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872db28-ab7b-4175-aecb-55d54092809a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.465026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.465643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.466012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.466240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.466796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.469074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.469726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7b6c5-b910-4bf0-b454-1ce55390a75d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.478743 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.480030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.480326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.480607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.481317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.484130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.494158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:26.502293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1021/2061] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:26.507465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.513447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1041/2062] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:26.514555 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.527984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad690-7f9d-4479-85a0-fe75e8ef8a28 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.593544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:26.594510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a5d6b-522a-44fb-8788-4b6dc6df5a5f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.600676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bbb4c-b739-4855-b5f7-1494cfb6eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.601477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bbb4c-b739-4855-b5f7-1494cfb6eab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.601781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bbb4c-b739-4855-b5f7-1494cfb6eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.602039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bbb4c-b739-4855-b5f7-1494cfb6eab2 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.602670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bbb4c-b739-4855-b5f7-1494cfb6eab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.602872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.603531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.603809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.604114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] PATH_INFO: `/v3/projects/08153f05fd3d46b0bd8bfd67fef6de40/users/9eda944896e440009ced22ef6732acb7/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.604661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bbb4c-b739-4855-b5f7-1494cfb6eab2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.606092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08153f05fd3d46b0bd8bfd67fef6de40, user_id=9eda944896e440009ced22ef6732acb7, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.608761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.620739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1022/2063] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:26 2022] PUT /identity/v3/projects/08153f05fd3d46b0bd8bfd67fef6de40/users/9eda944896e440009ced22ef6732acb7/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:26.621550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.623239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.631305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1042/2064] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:26.632926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.633422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.633944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.668030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] MFA Rules not processed for user `9eda944896e440009ced22ef6732acb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:26.669285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.697927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb5338-541b-4e1c-88f7-5abbe24f73f8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.703465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.704213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.704501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.704783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] PATH_INFO: `/v3/users/32b5440737524485bcb815cf05949995` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.705981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b5440737524485bcb815cf05949995)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.708736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.748777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1043/2065] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:26.752935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.762795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b5440737524485bcb815cf05949995'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.765604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264be25d-69fe-4830-990e-376a915f0a4b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.771597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b5440737524485bcb815cf05949995'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.771741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.772538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.772973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.773254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.774694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.775606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.788419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b5440737524485bcb815cf05949995'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.789638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b5440737524485bcb815cf05949995'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.790749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1044/2066] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:26.791793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.793575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.802824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b5440737524485bcb815cf05949995'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:26.809765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1023/2067] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:26 2022] DELETE /identity/v3/users/32b5440737524485bcb815cf05949995 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:26.810679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.812520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.864490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eeada-d546-4882-b9b2-af8a2b2af742 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.878297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:26.882989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc182df-7431-4f62-8781-c72b0f3fdac7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.888466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10cf01-e318-4caa-9534-fe6738106d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.889080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cf01-e318-4caa-9534-fe6738106d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.889358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cf01-e318-4caa-9534-fe6738106d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.889626 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10cf01-e318-4caa-9534-fe6738106d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.890266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10cf01-e318-4caa-9534-fe6738106d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.891850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10cf01-e318-4caa-9534-fe6738106d67 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.901981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1024/2068] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:26.905316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439b82-4704-4273-8170-1965b80c98ec None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.905604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439b82-4704-4273-8170-1965b80c98ec None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.905881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439b82-4704-4273-8170-1965b80c98ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.914359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:26.918405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.919014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.919293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.919569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.921479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.926775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-55439b82-4704-4273-8170-1965b80c98ec None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:26.928193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-55439b82-4704-4273-8170-1965b80c98ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.935716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1025/2069] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:26.937079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55439b82-4704-4273-8170-1965b80c98ec None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.947205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55439b82-4704-4273-8170-1965b80c98ec None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:26.951915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:26.952590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:26.952887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:26.953153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] PATH_INFO: `/v3/projects/576b7062b85541d0a5e805db91195de2/users/dfa980de381d4560b9fcde18b503bfc6/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:26.968962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=576b7062b85541d0a5e805db91195de2, user_id=dfa980de381d4560b9fcde18b503bfc6, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.971647 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:26.985268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1026/2070] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:26 2022] PUT /identity/v3/projects/576b7062b85541d0a5e805db91195de2/users/dfa980de381d4560b9fcde18b503bfc6/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:26.986053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:26.991862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:26.995367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:26.996871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e12a3-af2c-457d-bc40-9a4c39ef8419 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.008180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1045/2071] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:27.009531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.009753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.010029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.043193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] MFA Rules not processed for user `dfa980de381d4560b9fcde18b503bfc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:27.045210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.059485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bc241-6c49-4495-94fb-e3c586210116 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.063302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.064037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.064324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.064596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.138931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.140657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.150411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1027/2072] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:27.151284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.152856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.154844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1046/2073] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:27.160899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.171907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d594b-8311-4972-bbde-2da751672409 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.178059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.178676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.178959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.179238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.179876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.181854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.213242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1047/2074] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:27 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:27.217177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.227831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e053b-b342-4513-a3b2-0e8d2e99e5bf service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.262231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:27.280882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:27.292740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c881e9-0820-485f-bafc-efaf4219cc87 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.297866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.298298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.299142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.299424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.302958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.303912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.304579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.304914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.305198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.305892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.308296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.339371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1048/2075] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:27 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:27.342024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.356410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb4b65-25b4-4607-8d20-aca5a52d17d3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.362822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.364491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.364896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.365459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.366355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.368250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.386134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1049/2076] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:27.389576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.389767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.390512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.400343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1028/2077] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:27.415790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2ec9-ada5-42f7-84dd-ffbc894d4d14 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.421993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.422678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.422964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.423201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] PATH_INFO: `/v3/projects/a89447b32a794a3d8f4790242ae39dfa/users/e647960f5aeb4b7f8063a0f764d7748f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.452985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.454535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.463024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89447b32a794a3d8f4790242ae39dfa, user_id=e647960f5aeb4b7f8063a0f764d7748f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.473306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e928a-27e0-4a42-a955-80941ef5fbb3 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.479138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.479855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.480106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.480528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] PATH_INFO: `/v3/projects/beb67fcf6f8f4b16964737fc623508cc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.481885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.482402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb67fcf6f8f4b16964737fc623508cc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.485029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.505056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1050/2078] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:27 2022] PUT /identity/v3/projects/a89447b32a794a3d8f4790242ae39dfa/users/e647960f5aeb4b7f8063a0f764d7748f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:27.509897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.546364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb67fcf6f8f4b16964737fc623508cc', 'request_id': 'req-48fa3232-9ebd-42b4-a036-caac85a2d7ad'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.554565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb67fcf6f8f4b16964737fc623508cc', 'request_id': 'req-48fa3232-9ebd-42b4-a036-caac85a2d7ad'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.561501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb67fcf6f8f4b16964737fc623508cc', 'request_id': 'req-48fa3232-9ebd-42b4-a036-caac85a2d7ad'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.585183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784f55c-9e1d-4e97-b7bf-19cf9b972593 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.586602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] The token cache is being invalidate because project beb67fcf6f8f4b16964737fc623508cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:27.587044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.587833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.590580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1029/2079] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:27 2022] DELETE /identity/v3/projects/beb67fcf6f8f4b16964737fc623508cc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:27.595521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.597302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.665266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.666146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.666497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.666844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.667681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.669461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.681282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1051/2080] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:27.689574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.693279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.694241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3232-9ebd-42b4-a036-caac85a2d7ad admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.699209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.699712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.700018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.700292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] PATH_INFO: `/v3/users/d5ae9dd8de934c64ad5a85aebae3505a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.701411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ae9dd8de934c64ad5a85aebae3505a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.703331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.706983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5d4ef-fbec-471f-8fb1-b3bd2951892a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.712764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71d469-d3d5-4e70-a0e4-e2d0b2377103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.713421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d469-d3d5-4e70-a0e4-e2d0b2377103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.713698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d469-d3d5-4e70-a0e4-e2d0b2377103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.713972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d469-d3d5-4e70-a0e4-e2d0b2377103 admin admin] PATH_INFO: `/v3/projects/a89447b32a794a3d8f4790242ae39dfa/users/e647960f5aeb4b7f8063a0f764d7748f/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.715873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71d469-d3d5-4e70-a0e4-e2d0b2377103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89447b32a794a3d8f4790242ae39dfa, user_id=e647960f5aeb4b7f8063a0f764d7748f, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:27.718957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71d469-d3d5-4e70-a0e4-e2d0b2377103 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:27.729176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:27.730388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1052/2081] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:27 2022] PUT /identity/v3/projects/a89447b32a794a3d8f4790242ae39dfa/users/e647960f5aeb4b7f8063a0f764d7748f/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:27.740783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.741209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.741625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:27.760442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ae9dd8de934c64ad5a85aebae3505a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.777279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] MFA Rules not processed for user `e647960f5aeb4b7f8063a0f764d7748f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:27.778580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ae9dd8de934c64ad5a85aebae3505a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.778988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.788488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:27.811027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ae9dd8de934c64ad5a85aebae3505a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.812376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ae9dd8de934c64ad5a85aebae3505a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.825922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ae9dd8de934c64ad5a85aebae3505a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:27.833853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1030/2082] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:27 2022] DELETE /identity/v3/users/d5ae9dd8de934c64ad5a85aebae3505a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:27.884048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1053/2083] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:27.896649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:27.898112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:27.969480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d533632-79a3-4823-9702-c36848cd4af8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:27.973847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:27.974474 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:27.974760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:27.975019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.069040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.070612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.083464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1031/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:28.180805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.183124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.214392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.216945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.282965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444688e-f220-4353-af64-92c160ee81c3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.287567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.288206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.288487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.288761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.290465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.298924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:28.318804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6dc8-4f60-43ea-82b4-5f87393a4258 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.323554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.324291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.324645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.324993 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] PATH_INFO: `/v3/projects/bccb1f5178c6486bbc48d58e703e48df` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.326196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccb1f5178c6486bbc48d58e703e48df)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.328212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.352157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.353585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0f269-1815-4685-8313-70dd52bc99e7 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.363103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1054/2085] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:28.379089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccb1f5178c6486bbc48d58e703e48df', 'request_id': 'req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:28.388641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccb1f5178c6486bbc48d58e703e48df', 'request_id': 'req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:28.396053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccb1f5178c6486bbc48d58e703e48df', 'request_id': 'req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:28.418837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] The token cache is being invalidate because project bccb1f5178c6486bbc48d58e703e48df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:28.418837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:28.418837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f94a5b0b-bc44-44f3-8a2e-da1e0213ce75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:28.418837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1032/2086] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:28 2022] DELETE /identity/v3/projects/bccb1f5178c6486bbc48d58e703e48df => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:28.419190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.419408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.419685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.439566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:28.441040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.494504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:28.510167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1055/2087] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:28.518111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.518344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.518639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.541206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:28.542438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:28.550604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1033/2088] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:28.555243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.566066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a719b-b1dc-4ce8-88d9-7451c18da53b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.572487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.573236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.573521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.573832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.574462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:28.574772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.576738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.590834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1056/2089] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:40:28.596264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.607427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b33848-998f-47ae-8302-5670b1ef177a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.612397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.613081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.613580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.613859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.614827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.616967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.671497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1034/2090] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:28 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:28.675320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.702298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051157-fb38-44f7-b625-d040486eba65 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.757329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.757940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.758223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.758499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.759128 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.761083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.784842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1057/2091] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:28 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:28.789016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.799567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4d0f1-2303-49c3-b175-c348da3f69e5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.804478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.805090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.805375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.805647 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.806309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.807901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.818675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1035/2092] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:28.822572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.833381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925121-18bb-40fb-a24b-391abe5f02ea admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.838059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b509665-27c8-4c13-8325-b3fc3c209971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.838688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b509665-27c8-4c13-8325-b3fc3c209971 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.838971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b509665-27c8-4c13-8325-b3fc3c209971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.839249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b509665-27c8-4c13-8325-b3fc3c209971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.839892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b509665-27c8-4c13-8325-b3fc3c209971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.841505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b509665-27c8-4c13-8325-b3fc3c209971 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.852642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1058/2093] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:28.857012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:28.867995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac24850-17e3-4d64-8aae-da280975be7c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:28.872661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:28.873334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.873639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.873963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] PATH_INFO: `/v3/projects/c2a40408bc58405a8cc5e90a35fbc72b/users/e8701ca2fd334c4db5ba07b9f3e4b76c/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.889632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a40408bc58405a8cc5e90a35fbc72b, user_id=e8701ca2fd334c4db5ba07b9f3e4b76c, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:28.892464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:28.905804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1036/2094] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:28 2022] PUT /identity/v3/projects/c2a40408bc58405a8cc5e90a35fbc72b/users/e8701ca2fd334c4db5ba07b9f3e4b76c/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:28.917289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:28.917582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:28.917868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:28.945756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] MFA Rules not processed for user `e8701ca2fd334c4db5ba07b9f3e4b76c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:28.946980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:29.013342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1059/2095] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:29.017910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.083933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce78883-3400-4cba-93a7-68d7ca64bb07 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.088884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.089495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.089776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.090056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.090709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.092692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.113381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:29.124117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1037/2096] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:29 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:29.126524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.138520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5617c2-fbb4-4ec3-96dd-e6e3f01991d8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.190353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:29.199315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.199921 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.200491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.200761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.201389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.203384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.224571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1060/2097] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:29 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:29.228529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.239257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995f0b6-99fa-42b5-a5fb-72a8fe472fbc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.245861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.246720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.247055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.247576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.248353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.250122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.264426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1038/2098] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:29.268569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.284324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be13d-08f0-4a10-b854-d902641f6f7b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.289848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.291288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.291763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.292220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] PATH_INFO: `/v3/projects/ca30302b8eb0410fafee87cc621bba65/users/be49cdf5e6ab4d14ae9d8709578aae45/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.309592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca30302b8eb0410fafee87cc621bba65, user_id=be49cdf5e6ab4d14ae9d8709578aae45, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.312264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.326262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1061/2099] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:29 2022] PUT /identity/v3/projects/ca30302b8eb0410fafee87cc621bba65/users/be49cdf5e6ab4d14ae9d8709578aae45/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:29.330311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.341277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e81e7-12ee-4b3a-adf1-ace367535222 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.401652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb84e8f-3bb0-44dd-b1af-54902538338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.402261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb84e8f-3bb0-44dd-b1af-54902538338d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.402574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb84e8f-3bb0-44dd-b1af-54902538338d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.402872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb84e8f-3bb0-44dd-b1af-54902538338d admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.403579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb84e8f-3bb0-44dd-b1af-54902538338d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.405475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb84e8f-3bb0-44dd-b1af-54902538338d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.416112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1039/2100] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:29.420323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.430903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eef9eb-cdf1-45c9-9a28-e06e03ad92f6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.435507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.436191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.436470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.436741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] PATH_INFO: `/v3/projects/ca30302b8eb0410fafee87cc621bba65/users/be49cdf5e6ab4d14ae9d8709578aae45/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.438690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca30302b8eb0410fafee87cc621bba65, user_id=be49cdf5e6ab4d14ae9d8709578aae45, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.442061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.452605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1062/2101] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:29 2022] PUT /identity/v3/projects/ca30302b8eb0410fafee87cc621bba65/users/be49cdf5e6ab4d14ae9d8709578aae45/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:29.463331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.463621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.463901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.494770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] MFA Rules not processed for user `be49cdf5e6ab4d14ae9d8709578aae45`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:29.496215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:29.579031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1040/2102] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:29.590846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.592675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:29.665776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582aef82-51f1-4c27-b6dd-dcd0344264dd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.669396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06fb69-4bce-4a4e-9d6b-56810e708537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.669997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06fb69-4bce-4a4e-9d6b-56810e708537 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.670273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06fb69-4bce-4a4e-9d6b-56810e708537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.670543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06fb69-4bce-4a4e-9d6b-56810e708537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.679879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06fb69-4bce-4a4e-9d6b-56810e708537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.681307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06fb69-4bce-4a4e-9d6b-56810e708537 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.690971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1063/2103] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:29.720968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:29.826709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8d3b-44b8-4f4c-bc18-e81aeb859025 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:29.831362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:29.832147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.832538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.832892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] PATH_INFO: `/v3/users/e8701ca2fd334c4db5ba07b9f3e4b76c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.834334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8701ca2fd334c4db5ba07b9f3e4b76c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:29.836262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:29.848805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:29.895107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8701ca2fd334c4db5ba07b9f3e4b76c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:29.905528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8701ca2fd334c4db5ba07b9f3e4b76c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:29.921142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8701ca2fd334c4db5ba07b9f3e4b76c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:29.921849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8701ca2fd334c4db5ba07b9f3e4b76c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:29.931580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8701ca2fd334c4db5ba07b9f3e4b76c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:29.938021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1041/2104] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:29 2022] DELETE /identity/v3/users/e8701ca2fd334c4db5ba07b9f3e4b76c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:29.944459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:29.944720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:29.944998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:29.966394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:29.967632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:30.064561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1064/2105] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:30.067027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:30.069218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:30.152403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3620c-e96e-4722-a553-2cda901b50a2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:30.162925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:30.163741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:30.164382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:30.164954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:30.177648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:30.179417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:30.191710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1042/2106] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:30.505849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:30.508806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:30.527523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb618c-acf6-4b0d-91cf-62b8379c9401 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:30.548618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:30.549408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:30.551973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:30.552360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] PATH_INFO: `/v3/projects/c2a40408bc58405a8cc5e90a35fbc72b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:30.553739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a40408bc58405a8cc5e90a35fbc72b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:30.555764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:30.613934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:30.616648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a40408bc58405a8cc5e90a35fbc72b', 'request_id': 'req-553ffbb2-e036-4ac7-b75f-3d0f87221793'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.625342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a40408bc58405a8cc5e90a35fbc72b', 'request_id': 'req-553ffbb2-e036-4ac7-b75f-3d0f87221793'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.633706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a40408bc58405a8cc5e90a35fbc72b', 'request_id': 'req-553ffbb2-e036-4ac7-b75f-3d0f87221793'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.644996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] The token cache is being invalidate because project c2a40408bc58405a8cc5e90a35fbc72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:30.645404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.646140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.647673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1065/2107] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:30 2022] DELETE /identity/v3/projects/c2a40408bc58405a8cc5e90a35fbc72b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:30.651590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:30.661878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427e095-b19d-453d-baca-e05c5611c279 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:30.725372 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:30.726002 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:30.726274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:30.726542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] PATH_INFO: `/v3/users/be49cdf5e6ab4d14ae9d8709578aae45` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:30.727712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be49cdf5e6ab4d14ae9d8709578aae45)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:30.729754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:30.768860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:30.786064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be49cdf5e6ab4d14ae9d8709578aae45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.793876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be49cdf5e6ab4d14ae9d8709578aae45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.811824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be49cdf5e6ab4d14ae9d8709578aae45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.812721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be49cdf5e6ab4d14ae9d8709578aae45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.822942 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f0cba923-2937-42fd-b723-4440b2ea390b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be49cdf5e6ab4d14ae9d8709578aae45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:30.830791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1043/2108] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:30 2022] DELETE /identity/v3/users/be49cdf5e6ab4d14ae9d8709578aae45 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:31.130757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.132225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:31.149064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ffbb2-e036-4ac7-b75f-3d0f87221793 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.240105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.241089 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.241455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.241799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] PATH_INFO: `/v3/projects/ca30302b8eb0410fafee87cc621bba65` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.243070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca30302b8eb0410fafee87cc621bba65)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.245586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.306007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca30302b8eb0410fafee87cc621bba65', 'request_id': 'req-a4287337-1eab-485f-b285-60f50bf96261'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.318391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca30302b8eb0410fafee87cc621bba65', 'request_id': 'req-a4287337-1eab-485f-b285-60f50bf96261'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.327622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca30302b8eb0410fafee87cc621bba65', 'request_id': 'req-a4287337-1eab-485f-b285-60f50bf96261'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.340271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] The token cache is being invalidate because project ca30302b8eb0410fafee87cc621bba65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:31.340646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.341400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a4287337-1eab-485f-b285-60f50bf96261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.343526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1066/2109] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:31 2022] DELETE /identity/v3/projects/ca30302b8eb0410fafee87cc621bba65 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:31.351193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.351690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.351783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.374583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:31.375793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:31.459912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1044/2110] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:31.461668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.461941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.462222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.489607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:31.491039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:31.503708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1067/2111] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:31.505996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.517112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a74d7-36cc-4ac7-8a53-ad49c399b0ff None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.524828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.525874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.526172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.526638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.527676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:31.528085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.530231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.545658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1045/2112] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:31.550146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.566189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498354e-ab04-4d00-8596-70c46b88a5d2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.571066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.571789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.571932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.572316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.572963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.574893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.619983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1068/2113] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:31 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:31.624426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.642191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53085709-2bc3-4ba9-872d-b54b7a156ca3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.772716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.774980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.775679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.776033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.776353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.777178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.778333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:31.779682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.801746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdbd6e-882e-4ce7-8469-11a7f31febf9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.807094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.807296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.807668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.808142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] PATH_INFO: `/v3/users/2319cf8b699d4ffb8dc907dade820b20` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.809799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2319cf8b699d4ffb8dc907dade820b20)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.812155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.818321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1046/2114] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:31 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:31.822989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.840805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31d60-1a79-4866-9a88-d27047089d5b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.849557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.850321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.850818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.851898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.852810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.856527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.857567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:31.876901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1047/2115] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:31.877901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2319cf8b699d4ffb8dc907dade820b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.878446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.888145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2319cf8b699d4ffb8dc907dade820b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.890366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9758d77-92c1-4401-b5d9-e690bd8190c0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.895737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2319cf8b699d4ffb8dc907dade820b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.901916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:31.902734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.902982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.903377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.904777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:31.906699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2319cf8b699d4ffb8dc907dade820b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.906832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:31.917873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1048/2116] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:31.923477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:31.925212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-764dc92b-278a-49a7-9fc3-b10c0195e988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2319cf8b699d4ffb8dc907dade820b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:31.925372 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:31.932872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1069/2117] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:31 2022] DELETE /identity/v3/users/2319cf8b699d4ffb8dc907dade820b20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:31.937973 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:31.938269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:31.938538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:31.939274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:31.946890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d3ddf-4593-4e69-940b-8527318a385b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:31.964279 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:31.966132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:32.015040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebf17c-ff52-4c81-b119-a083d14b012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:32.015621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebf17c-ff52-4c81-b119-a083d14b012d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:32.015911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebf17c-ff52-4c81-b119-a083d14b012d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:32.016222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebf17c-ff52-4c81-b119-a083d14b012d admin admin] PATH_INFO: `/v3/projects/a999efed32524f878ccbb21215d3f3cd/users/412399f7b74b48d685f8ba7b305ad6a8/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:32.042488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1070/2118] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:32.042910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebf17c-ff52-4c81-b119-a083d14b012d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a999efed32524f878ccbb21215d3f3cd, user_id=412399f7b74b48d685f8ba7b305ad6a8, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:32.046769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebf17c-ff52-4c81-b119-a083d14b012d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:32.053262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:32.054823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:32.065222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1049/2119] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:31 2022] PUT /identity/v3/projects/a999efed32524f878ccbb21215d3f3cd/users/412399f7b74b48d685f8ba7b305ad6a8/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:32.071490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:32.072051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:32.072639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:32.102797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:32.104593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:32.133156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2712aed-18cf-46d2-bcfe-35c858dd1028 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:32.204485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1050/2120] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:32.213482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:32.214132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:32.214242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:32.214417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:32.214821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:32.227731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:32.229357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:32.231264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051420b-da1a-4da1-ae23-e5aa4ee36bca None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:32.243502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1071/2121] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:32.256728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:32.257346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:32.257630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:32.257902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:32.279566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:32.281642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:32.306723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1051/2122] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:32.665303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:32.667519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:32.700194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8d0eb-7cde-42ee-be62-154cb58ad2f2 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:32.713168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:32.714326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:32.714755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:32.715514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] PATH_INFO: `/v3/projects/e91956bfef8d47c7a47b71e39d8160ce` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:32.717015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91956bfef8d47c7a47b71e39d8160ce)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:32.719589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:32.721446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:32.829941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91956bfef8d47c7a47b71e39d8160ce', 'request_id': 'req-3f7315b1-545c-4569-8f05-95c4c46313e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:32.838231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91956bfef8d47c7a47b71e39d8160ce', 'request_id': 'req-3f7315b1-545c-4569-8f05-95c4c46313e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:32.848912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91956bfef8d47c7a47b71e39d8160ce', 'request_id': 'req-3f7315b1-545c-4569-8f05-95c4c46313e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:32.862726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] The token cache is being invalidate because project e91956bfef8d47c7a47b71e39d8160ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:32.864228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:32.878005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f7315b1-545c-4569-8f05-95c4c46313e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:32.882710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1072/2123] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:32 2022] DELETE /identity/v3/projects/e91956bfef8d47c7a47b71e39d8160ce => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:32.887047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:32.918838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddf459-5090-4eee-b8d6-851e8d6f1e76 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:33.047262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:33.048362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.048618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.048955 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] PATH_INFO: `/v3/users/dccbec7f1a0e48d480126eb4bc663220` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.050742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dccbec7f1a0e48d480126eb4bc663220)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:33.055778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:33.074692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:33.119338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dccbec7f1a0e48d480126eb4bc663220'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.134190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dccbec7f1a0e48d480126eb4bc663220'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.150405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dccbec7f1a0e48d480126eb4bc663220'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.151200 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dccbec7f1a0e48d480126eb4bc663220'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.159326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dccbec7f1a0e48d480126eb4bc663220'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.167246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1052/2124] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:32 2022] DELETE /identity/v3/users/dccbec7f1a0e48d480126eb4bc663220 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:33.421519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.422122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.422580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.434112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:33.435569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.465570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] MFA Rules not processed for user `412399f7b74b48d685f8ba7b305ad6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:33.469472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.563698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4b4a0-82b6-435a-8570-dc42221fcad3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:33.571040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:33.571988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.572769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.588101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] PATH_INFO: `/v3/users/2200bb80b6634fb69db1570ef64ff714` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.597612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2200bb80b6634fb69db1570ef64ff714)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:33.602316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:33.624596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1073/2125] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:33.624596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:33.626699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.657902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9424bd4-2d32-451f-9c23-10c0948eb29e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:33.672403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:33.675272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.676988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.678369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] PATH_INFO: `/v3/projects/6fe2f942e1b54a349b448ba4f3d43d2d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.682695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe2f942e1b54a349b448ba4f3d43d2d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:33.686932 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:33.700445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2200bb80b6634fb69db1570ef64ff714'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.717897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2200bb80b6634fb69db1570ef64ff714'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.733483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2200bb80b6634fb69db1570ef64ff714'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.734544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2200bb80b6634fb69db1570ef64ff714'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.743124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2200bb80b6634fb69db1570ef64ff714'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.752195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1053/2126] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:33 2022] DELETE /identity/v3/users/2200bb80b6634fb69db1570ef64ff714 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:33.752195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:33.754466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.773426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe2f942e1b54a349b448ba4f3d43d2d', 'request_id': 'req-0be00eba-f53e-4f11-b36f-cb40e988834e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.775301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:33.780906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe2f942e1b54a349b448ba4f3d43d2d', 'request_id': 'req-0be00eba-f53e-4f11-b36f-cb40e988834e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.791568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe2f942e1b54a349b448ba4f3d43d2d', 'request_id': 'req-0be00eba-f53e-4f11-b36f-cb40e988834e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.819824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] The token cache is being invalidate because project 6fe2f942e1b54a349b448ba4f3d43d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:33.820172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.820862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0be00eba-f53e-4f11-b36f-cb40e988834e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.822398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1074/2127] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:33 2022] DELETE /identity/v3/projects/6fe2f942e1b54a349b448ba4f3d43d2d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:33.825532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.825532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.825532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.834338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:33.852339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c6365-eb37-4942-b13d-357e42b2783d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:33.852626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:33.854521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.857823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:33.858613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.859054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.859241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.860034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:33.862290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:33.862871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1075/2128] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:33.864436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:33.865807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.877844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c57d7f-05e6-4877-b964-3ef1c704b8f3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:33.883022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:33.883655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.883937 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.884239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] PATH_INFO: `/v3/users/9eda944896e440009ced22ef6732acb7` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.885863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eda944896e440009ced22ef6732acb7)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:33.887822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:33.899151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1054/2129] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:33 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:33.903445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:33.905569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:33.941237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eda944896e440009ced22ef6732acb7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.950515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eda944896e440009ced22ef6732acb7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.951519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eda944896e440009ced22ef6732acb7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.960799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eda944896e440009ced22ef6732acb7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.975390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eda944896e440009ced22ef6732acb7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:33.977231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465de89d-c609-43e5-803d-96729b4108ec admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:33.981848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:33.981980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1076/2130] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:33 2022] DELETE /identity/v3/users/9eda944896e440009ced22ef6732acb7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:33.982476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:33.982709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:33.983274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:33.983699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] PATH_INFO: `/v3/users/dfa980de381d4560b9fcde18b503bfc6` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:33.985226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa980de381d4560b9fcde18b503bfc6)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:33.987473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:33.990081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.046331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa980de381d4560b9fcde18b503bfc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.059404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa980de381d4560b9fcde18b503bfc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.083327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa980de381d4560b9fcde18b503bfc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.084396 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa980de381d4560b9fcde18b503bfc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.093155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa980de381d4560b9fcde18b503bfc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.101449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1055/2131] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:33 2022] DELETE /identity/v3/users/dfa980de381d4560b9fcde18b503bfc6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:34.102771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.104463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.126181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:34.138675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8fb80-ac55-49db-8620-c8e289546f69 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.201144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebafd7-4e58-4e6f-b7c3-c1d2734f2ecc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.207373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6789b44-0b17-421d-838a-739bc7dce9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.208284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789b44-0b17-421d-838a-739bc7dce9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.208757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789b44-0b17-421d-838a-739bc7dce9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.209216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789b44-0b17-421d-838a-739bc7dce9fb admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.210311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789b44-0b17-421d-838a-739bc7dce9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.213055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789b44-0b17-421d-838a-739bc7dce9fb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.229784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.231104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.231278 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.232105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.233996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.243096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1056/2132] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:34 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:34.244494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.244782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.245059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.260223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.262774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.266254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:34.267465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.277310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1077/2133] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:34.278160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.282351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1057/2134] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:34.292443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a0b9-ffe2-49ef-a46c-499f90f4bf19 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.293074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.297062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.298050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.298755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.299286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.299644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.300437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.303076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.311339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f8afd-d8e9-43d2-a41e-8c31c2acf544 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.315833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.316527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.316805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.317092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.317894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1078/2135] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:34.328585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.337769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.341196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.354386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0b1f-7738-4dc8-a1d9-55c938d516c0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.368884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.369689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.370104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.370584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1058/2136] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:34.370850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] PATH_INFO: `/v3/projects/a4b039bb36fd4a44968e5f71a8dea2d3/users/1b075b4f67f345afa7d42b336815f6c1/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.390017 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:34.397528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b039bb36fd4a44968e5f71a8dea2d3, user_id=1b075b4f67f345afa7d42b336815f6c1, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.401491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.421385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.423263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.431669 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1079/2137] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:34 2022] PUT /identity/v3/projects/a4b039bb36fd4a44968e5f71a8dea2d3/users/1b075b4f67f345afa7d42b336815f6c1/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:34.449818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.461534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a0ef9-dd37-43dd-bed1-cc5f13074127 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.588400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.590087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.590477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.590861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] PATH_INFO: `/v3/projects/08153f05fd3d46b0bd8bfd67fef6de40` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.592444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08153f05fd3d46b0bd8bfd67fef6de40)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.595054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.620041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700df87-ea73-4af7-99c8-61cda78b82d8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.625666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.625916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.626324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.632439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.633495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.635599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.655996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1080/2138] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:34.659737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.671598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe1704-ac1d-4d5a-9a18-ac2489698aaa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:34.672025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08153f05fd3d46b0bd8bfd67fef6de40', 'request_id': 'req-331f6470-916d-4fe7-a34c-ba8b97a3424e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.679517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08153f05fd3d46b0bd8bfd67fef6de40', 'request_id': 'req-331f6470-916d-4fe7-a34c-ba8b97a3424e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.688484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08153f05fd3d46b0bd8bfd67fef6de40', 'request_id': 'req-331f6470-916d-4fe7-a34c-ba8b97a3424e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.711912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] The token cache is being invalidate because project 08153f05fd3d46b0bd8bfd67fef6de40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:34.712451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.713328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-331f6470-916d-4fe7-a34c-ba8b97a3424e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:34.715229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1059/2139] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:34 2022] DELETE /identity/v3/projects/08153f05fd3d46b0bd8bfd67fef6de40 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:34.721426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.721776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.722121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.750222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:34.752075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.764300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad0793-9753-4ff4-98b6-1ac1487f0a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:34.765663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad0793-9753-4ff4-98b6-1ac1487f0a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.766227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad0793-9753-4ff4-98b6-1ac1487f0a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.766517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad0793-9753-4ff4-98b6-1ac1487f0a3e admin admin] PATH_INFO: `/v3/projects/a4b039bb36fd4a44968e5f71a8dea2d3/users/1b075b4f67f345afa7d42b336815f6c1/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.768004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:34.771535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad0793-9753-4ff4-98b6-1ac1487f0a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b039bb36fd4a44968e5f71a8dea2d3, user_id=1b075b4f67f345afa7d42b336815f6c1, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:34.776632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad0793-9753-4ff4-98b6-1ac1487f0a3e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:34.797656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1081/2140] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:34 2022] PUT /identity/v3/projects/a4b039bb36fd4a44968e5f71a8dea2d3/users/1b075b4f67f345afa7d42b336815f6c1/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:34.856030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1060/2141] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:34.860538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:34.860909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:34.861101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:34.887594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:34.889913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:34.975907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1082/2142] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:34.980149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:34.993583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26637bfa-c1a8-4630-9ce3-99f253579ce7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.006551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.007418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.007829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.009139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.009639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:35.009950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.013457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.030274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1061/2143] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:35.034525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.050378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fbfa4-7617-4b98-9de5-a526b11f71e2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.056295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.057116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.057389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.057728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.058371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.060894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.125360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:35.146208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1083/2144] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:35 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:35.147512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.169810 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.173656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9416e-b5ef-4583-beb5-063223874cac admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.179262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:35.368749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.369373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.371345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.372321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.372525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.377545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.383001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6af1c-18b5-4f51-8d86-81c909b4d344 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.390692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.391985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.393907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.397090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] PATH_INFO: `/v3/projects/35e653077b054b33bb90d5d7918ef4c4` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.399091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e653077b054b33bb90d5d7918ef4c4)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.401832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.430583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1062/2145] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:35 2022] POST /identity/v3/users => generated 287 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:35.432441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.455315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66b8d-e4f2-4df1-9ad0-26d8a1d03926 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.475689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.476798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.477003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.477246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.478344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.480229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.521531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e653077b054b33bb90d5d7918ef4c4', 'request_id': 'req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.529347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1063/2146] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:35.532951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.540515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e653077b054b33bb90d5d7918ef4c4', 'request_id': 'req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.558465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e653077b054b33bb90d5d7918ef4c4', 'request_id': 'req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.564501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9528b-a727-443b-9261-354000067bcf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.589816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] The token cache is being invalidate because project 35e653077b054b33bb90d5d7918ef4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:35.590163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.592026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.594219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1084/2147] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:35 2022] DELETE /identity/v3/projects/35e653077b054b33bb90d5d7918ef4c4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:35.599622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.601938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:35.706097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.708822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.711792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.711792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.719543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.723170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.749329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1064/2148] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:35.751262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.754292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:35.771797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e04d-4318-41c3-aa00-c48bc12a8056 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.779118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.779772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.780180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.780547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] PATH_INFO: `/v3/projects/576b7062b85541d0a5e805db91195de2` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.781814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] RBAC: Authorizing `identity:delete_project(project_id=576b7062b85541d0a5e805db91195de2)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.782605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fd5ab-e29c-4ff1-84d3-57ec5a267ac3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:35.786259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.793455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:35.794351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:35.794781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:35.795096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] PATH_INFO: `/v3/users/2b56fbcc126947c09167cd5702f9cd7f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:35.809853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:35.812849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b56fbcc126947c09167cd5702f9cd7f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:35.832590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:35.878163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '576b7062b85541d0a5e805db91195de2', 'request_id': 'req-9c620a23-f56e-4edd-8bb2-5c386bf2640c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.886247 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '576b7062b85541d0a5e805db91195de2', 'request_id': 'req-9c620a23-f56e-4edd-8bb2-5c386bf2640c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.895545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '576b7062b85541d0a5e805db91195de2', 'request_id': 'req-9c620a23-f56e-4edd-8bb2-5c386bf2640c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.919008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] The token cache is being invalidate because project 576b7062b85541d0a5e805db91195de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:35.919439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.921472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.926034 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:35.930720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1065/2149] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:35 2022] DELETE /identity/v3/projects/576b7062b85541d0a5e805db91195de2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:35.936781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:35.942963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:35.950500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b56fbcc126947c09167cd5702f9cd7f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.958938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b56fbcc126947c09167cd5702f9cd7f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.960002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b56fbcc126947c09167cd5702f9cd7f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:35.979800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b56fbcc126947c09167cd5702f9cd7f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.011091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2850b244-92fd-4d0f-8ea0-4a08eda96b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b56fbcc126947c09167cd5702f9cd7f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.028880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1085/2150] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:35 2022] DELETE /identity/v3/users/2b56fbcc126947c09167cd5702f9cd7f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:36.046469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.053274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.053481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.117175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c620a23-f56e-4edd-8bb2-5c386bf2640c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:36.123865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] MFA Rules not processed for user `1b075b4f67f345afa7d42b336815f6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:36.128336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.290925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:36.292846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.293297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.293886 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] PATH_INFO: `/v3/projects/8424741e1d154c6c9b08b19fb7c61d33/users/e4b36e6959444273872c300742d6ddb8/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.325872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1086/2151] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:36.326611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:36.328995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.356808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8424741e1d154c6c9b08b19fb7c61d33, user_id=e4b36e6959444273872c300742d6ddb8, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:36.359595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:36.364805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab84385-5fa6-4a7c-a220-b829f1f2601b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:36.385023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:36.387541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.387845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.388338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] PATH_INFO: `/v3/users/e647960f5aeb4b7f8063a0f764d7748f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.390306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e647960f5aeb4b7f8063a0f764d7748f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:36.392692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:36.407307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1066/2152] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:35 2022] PUT /identity/v3/projects/8424741e1d154c6c9b08b19fb7c61d33/users/e4b36e6959444273872c300742d6ddb8/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:36.409060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:36.427532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.548274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:36.573593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e647960f5aeb4b7f8063a0f764d7748f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.583587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e647960f5aeb4b7f8063a0f764d7748f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.584468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e647960f5aeb4b7f8063a0f764d7748f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.587981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529c3b2-eb8e-421c-9253-5bbf6bbb9b37 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:36.592313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:36.592925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.593303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.593549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.595721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.597112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e647960f5aeb4b7f8063a0f764d7748f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.620607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-aed27f69-2973-4bfb-addd-5130431afe71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e647960f5aeb4b7f8063a0f764d7748f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.638425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1087/2153] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:36 2022] DELETE /identity/v3/users/e647960f5aeb4b7f8063a0f764d7748f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:36.640154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.640321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.640646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.664668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:36.672813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:36.675094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.704649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:36.707578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:36.726273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1067/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:36.727900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:36.729998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.773226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1088/2155] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:36.785929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:36.788957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:36.874211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753c26f-c1d5-4d91-8292-179e94b8359d service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:36.876959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b2eba-490b-430c-ae66-f062564a1fe0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:36.881390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7f201-41e4-4d78-9f14-cb4a5114a994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:36.882324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7f201-41e4-4d78-9f14-cb4a5114a994 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.882709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7f201-41e4-4d78-9f14-cb4a5114a994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.883049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7f201-41e4-4d78-9f14-cb4a5114a994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.889040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:36.890202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:36.890487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:36.890803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] PATH_INFO: `/v3/projects/5a21603a34314a5cba16d611993f4323` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:36.892671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a21603a34314a5cba16d611993f4323)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:36.894131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7f201-41e4-4d78-9f14-cb4a5114a994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:36.895797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7f201-41e4-4d78-9f14-cb4a5114a994 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:36.902007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:36.908385 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1089/2156] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:36.974694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a21603a34314a5cba16d611993f4323', 'request_id': 'req-2afab10a-fbeb-4118-bd37-a4281094bf77'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.983206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a21603a34314a5cba16d611993f4323', 'request_id': 'req-2afab10a-fbeb-4118-bd37-a4281094bf77'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:36.995078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a21603a34314a5cba16d611993f4323', 'request_id': 'req-2afab10a-fbeb-4118-bd37-a4281094bf77'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.015763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] The token cache is being invalidate because project 5a21603a34314a5cba16d611993f4323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:37.016418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.017305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2afab10a-fbeb-4118-bd37-a4281094bf77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.018878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1068/2157] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:36 2022] DELETE /identity/v3/projects/5a21603a34314a5cba16d611993f4323 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:37.024426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7580a-c151-4c90-b726-144d7010406c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.024982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7580a-c151-4c90-b726-144d7010406c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.025483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7580a-c151-4c90-b726-144d7010406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.052907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-15e7580a-c151-4c90-b726-144d7010406c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:37.054142 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-15e7580a-c151-4c90-b726-144d7010406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.128149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1090/2158] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:37.131547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.132098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.132098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.162437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:37.164380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.176901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1069/2159] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:37.181814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7580a-c151-4c90-b726-144d7010406c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.192764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.194026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7580a-c151-4c90-b726-144d7010406c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.196833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.205467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.207212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.207601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.208231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.209319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:37.209916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.211727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.212969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a6ec0-f64d-4637-b6d1-1349cf536bdc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.220284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.221097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.221377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.222930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] PATH_INFO: `/v3/projects/a89447b32a794a3d8f4790242ae39dfa` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.223820 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89447b32a794a3d8f4790242ae39dfa)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.226782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.236432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1091/2160] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:37.239046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:37.239543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.254291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f52859-f858-4b35-9cca-4e55f0b1d9ec admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.263349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.264176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.264550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.264880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.265556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.267798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.324814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1092/2161] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:37.326466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.351382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89447b32a794a3d8f4790242ae39dfa', 'request_id': 'req-122fec24-9195-43c5-8110-c92f93b9e6fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.370337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89447b32a794a3d8f4790242ae39dfa', 'request_id': 'req-122fec24-9195-43c5-8110-c92f93b9e6fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.373771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:37.388190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89447b32a794a3d8f4790242ae39dfa', 'request_id': 'req-122fec24-9195-43c5-8110-c92f93b9e6fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.408512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] The token cache is being invalidate because project a89447b32a794a3d8f4790242ae39dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:37.408988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.409932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-122fec24-9195-43c5-8110-c92f93b9e6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:37.413192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1070/2162] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:37 2022] DELETE /identity/v3/projects/a89447b32a794a3d8f4790242ae39dfa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:37.420697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1a879-c8dc-4d06-9127-c2e218ac2587 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.420916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1a879-c8dc-4d06-9127-c2e218ac2587 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.421196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1a879-c8dc-4d06-9127-c2e218ac2587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.499379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-57d1a879-c8dc-4d06-9127-c2e218ac2587 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:37.507565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-57d1a879-c8dc-4d06-9127-c2e218ac2587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.531261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090d2b-5636-4b68-aab3-bc0ce8341a5a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.550836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1071/2163] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:37.554711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.554711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.554877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.554976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.555050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.555120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.555187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.555255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.558426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.602863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:37.604293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.615197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1093/2164] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/users => generated 291 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:37.616653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.620984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.626010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1072/2165] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:37.636110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.671132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac256-a298-46f1-89ff-76aa96cffd79 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.674454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeae2ec-e246-4aa9-b4bb-fb06698afca1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.694994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.695902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.696564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.696925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.697458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.699301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.699599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.699883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.700587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.703147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:37.703604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.709465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.713901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.748342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1094/2166] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:37.754620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1073/2167] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:37.755104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.760007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.774736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb71efd-4118-4843-a46f-0def145a279e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.790940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.790940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.790940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.790940 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.791987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.791987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.792159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868fe84-cfc3-485c-9333-f5ab1b071209 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.794314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.794927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.795206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.795567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.796222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.798185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.801372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1095/2168] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:37.805520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.820391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49631ab-989d-41cf-ad16-52bba59c43fb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:37.828975 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777c87c-e080-4630-90de-8bcdb259c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:37.829833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c87c-e080-4630-90de-8bcdb259c12d admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.830788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c87c-e080-4630-90de-8bcdb259c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.831172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c87c-e080-4630-90de-8bcdb259c12d admin admin] PATH_INFO: `/v3/projects/dc2bc6a2ed9f43809e161d8aeb0cc955/users/541a4dd672294548a1715fe7bac417b1/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.846664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1074/2169] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:37.850591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777c87c-e080-4630-90de-8bcdb259c12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2bc6a2ed9f43809e161d8aeb0cc955, user_id=541a4dd672294548a1715fe7bac417b1, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:37.851401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:37.853879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777c87c-e080-4630-90de-8bcdb259c12d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:37.870824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1096/2170] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:37 2022] PUT /identity/v3/projects/dc2bc6a2ed9f43809e161d8aeb0cc955/users/541a4dd672294548a1715fe7bac417b1/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:37.883069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:37.883346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:37.883612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:37.917447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] MFA Rules not processed for user `541a4dd672294548a1715fe7bac417b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:37.918585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:37.935903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a9bc-f996-4f49-8be6-010dede5ed80 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.012208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1097/2171] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:38.013549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.017780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:38.052077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d975-016b-4446-a0d7-1b51e82c92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.053405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d975-016b-4446-a0d7-1b51e82c92b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.054071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d975-016b-4446-a0d7-1b51e82c92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.054071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d975-016b-4446-a0d7-1b51e82c92b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.055199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1d975-016b-4446-a0d7-1b51e82c92b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.056643 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1d975-016b-4446-a0d7-1b51e82c92b4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.087444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1075/2172] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:37 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:38.088809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.089084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.089382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.096443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:38.123072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] MFA Rules not processed for user `e4b36e6959444273872c300742d6ddb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:38.124217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba7c8-8c46-4830-8f18-88eaa5fc7d02 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.124330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.129906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.131119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.131259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.131440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.132124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.134078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.167644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1098/2173] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:38 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:38.168851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.206138 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1076/2174] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:38.208258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.233768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf6ada-2a2b-472f-96fa-0b8bbc3bbb7a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.247226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3808e0-5547-46bd-be19-539a22c13b76 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.252291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.252908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.253188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.253653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.254656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.256448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.271820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1099/2175] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:38.271820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.274673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.300538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.301753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.302090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.302454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.303179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.305335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.332867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1077/2176] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:38 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:38.332867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.346988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476ed99-321a-4ad1-8e0d-261e18eb7959 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.348594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e58aa-53fc-4556-a0cb-df2083b5e412 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.352337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.352953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.353236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.353532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.354000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.355880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.356201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.356469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.357112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.358972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.374190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1078/2177] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:38.374190 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.385288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe3133-fac7-4f80-80ca-c5c3ffaa075f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.390074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.390670 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.390946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.391219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.391854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.393512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.404446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1079/2178] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:38.408652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.415638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.418106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.419719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137944c-99f8-44ba-9d3c-82f47ab21c03 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.424276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ccb7f-a3ad-4fa6-ad23-79c302f0b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.424915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ccb7f-a3ad-4fa6-ad23-79c302f0b4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.425188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ccb7f-a3ad-4fa6-ad23-79c302f0b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.425556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ccb7f-a3ad-4fa6-ad23-79c302f0b4dd admin admin] PATH_INFO: `/v3/projects/cd6a3ffef6b3427893932c409e43994f/users/787451aea70c49d6814637d48037202f/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.428730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1100/2179] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:38.429728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.440859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e610bc-9f4f-47f3-8717-0badff4819e8 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.444238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ccb7f-a3ad-4fa6-ad23-79c302f0b4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6a3ffef6b3427893932c409e43994f, user_id=787451aea70c49d6814637d48037202f, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.445710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.446367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.446649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.446930 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] PATH_INFO: `/v3/projects/97c56f23aa1445058b9240738ad0e321/users/dad35ac7d9c847c4bd9792dfe922a3ce/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.447050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ccb7f-a3ad-4fa6-ad23-79c302f0b4dd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.461133 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1080/2180] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:38 2022] PUT /identity/v3/projects/cd6a3ffef6b3427893932c409e43994f/users/787451aea70c49d6814637d48037202f/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:38.464185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c56f23aa1445058b9240738ad0e321, user_id=dad35ac7d9c847c4bd9792dfe922a3ce, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.466903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.473203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.473613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.474119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.480080 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:38.487841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1101/2181] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:38 2022] PUT /identity/v3/projects/97c56f23aa1445058b9240738ad0e321/users/dad35ac7d9c847c4bd9792dfe922a3ce/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:38.493711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.519819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] MFA Rules not processed for user `787451aea70c49d6814637d48037202f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:38.521480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.528257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:38.586437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d411d2c-786e-4d5a-8163-348b84e64955 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.592380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.593120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.593408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.593733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.594340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.596520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.614296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1102/2182] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:38.614296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.614296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.623557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1081/2183] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:38.624875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.637206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f8e1a-c4b6-4b96-89af-84fd813398f7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.643072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a88d18-964b-4557-a986-ea1f039ea784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.643925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a88d18-964b-4557-a986-ea1f039ea784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.644985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a88d18-964b-4557-a986-ea1f039ea784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.645709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a88d18-964b-4557-a986-ea1f039ea784 admin admin] PATH_INFO: `/v3/projects/97c56f23aa1445058b9240738ad0e321/users/dad35ac7d9c847c4bd9792dfe922a3ce/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.648011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a88d18-964b-4557-a986-ea1f039ea784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c56f23aa1445058b9240738ad0e321, user_id=dad35ac7d9c847c4bd9792dfe922a3ce, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.651037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a88d18-964b-4557-a986-ea1f039ea784 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.670389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1082/2184] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:38 2022] PUT /identity/v3/projects/97c56f23aa1445058b9240738ad0e321/users/dad35ac7d9c847c4bd9792dfe922a3ce/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:38.683217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.683504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.683824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.688429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1cde-cbb2-41b6-83cb-0838faa2541d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.717681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] MFA Rules not processed for user `dad35ac7d9c847c4bd9792dfe922a3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:38.720257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.758484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.759114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.759437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.759712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.762249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:38.818285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1083/2185] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:38.819161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:38.837453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb12b-160e-405c-b909-c8663ad91fa7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:38.842771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:38.843644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:38.844204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:38.844611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:38.855802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.857509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.880378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1103/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:38.918508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:38.920303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:38.931128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1084/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:39.022153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:39.026372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:39.032925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849b3e0-54a9-4b5e-8cea-d1ae5f8f2dfa service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:39.039980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeeb44b-5434-4f3e-8002-46e8f8536d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:39.041143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb44b-5434-4f3e-8002-46e8f8536d3b service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.041499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb44b-5434-4f3e-8002-46e8f8536d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.041854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb44b-5434-4f3e-8002-46e8f8536d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.053556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeeb44b-5434-4f3e-8002-46e8f8536d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:39.055238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeeb44b-5434-4f3e-8002-46e8f8536d3b service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:39.065215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1104/2188] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:39.295597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:39.326432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:39.409347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86353183-df52-4bd2-b3da-65a6d154860e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:39.415116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:39.416027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.416312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.416591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] PATH_INFO: `/v3/users/541a4dd672294548a1715fe7bac417b1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.417745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=541a4dd672294548a1715fe7bac417b1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:39.419658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:39.483729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541a4dd672294548a1715fe7bac417b1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:39.494662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541a4dd672294548a1715fe7bac417b1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:39.509758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541a4dd672294548a1715fe7bac417b1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:39.510593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541a4dd672294548a1715fe7bac417b1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:39.519509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541a4dd672294548a1715fe7bac417b1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:39.525539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1085/2189] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:39 2022] DELETE /identity/v3/users/541a4dd672294548a1715fe7bac417b1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:39.530296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.530719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.531012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.550232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:39.552595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:39.619485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1105/2190] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:39.629889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:39.631325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:39.701275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b51ec6-0881-4605-b919-653ba3d53580 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:39.704898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef73a74-b4d7-474c-ab94-b0420cdcf923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:39.705744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73a74-b4d7-474c-ab94-b0420cdcf923 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.706028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73a74-b4d7-474c-ab94-b0420cdcf923 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.706302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef73a74-b4d7-474c-ab94-b0420cdcf923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.717760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef73a74-b4d7-474c-ab94-b0420cdcf923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:39.719199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef73a74-b4d7-474c-ab94-b0420cdcf923 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:39.732022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1086/2191] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:40.007837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.010026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.026455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f74349-dd16-4363-a302-c823f82b74c5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.039318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.040122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.040501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.040874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] PATH_INFO: `/v3/projects/dc2bc6a2ed9f43809e161d8aeb0cc955` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.042330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2bc6a2ed9f43809e161d8aeb0cc955)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.044939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.087388 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.087841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.088212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.104889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2bc6a2ed9f43809e161d8aeb0cc955', 'request_id': 'req-b0e4e15b-7852-484e-bea1-de87a2466db1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.109402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] MFA Rules not processed for user `e4b36e6959444273872c300742d6ddb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:40.110818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.111274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2bc6a2ed9f43809e161d8aeb0cc955', 'request_id': 'req-b0e4e15b-7852-484e-bea1-de87a2466db1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.121585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2bc6a2ed9f43809e161d8aeb0cc955', 'request_id': 'req-b0e4e15b-7852-484e-bea1-de87a2466db1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.134269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] The token cache is being invalidate because project dc2bc6a2ed9f43809e161d8aeb0cc955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:40.134677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.135484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.137145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1106/2192] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:40 2022] DELETE /identity/v3/projects/dc2bc6a2ed9f43809e161d8aeb0cc955 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:40.141222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.142626 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.150255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:40.181812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1087/2193] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:40.219878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4e15b-7852-484e-bea1-de87a2466db1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.225476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.226197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.226559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.227127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] PATH_INFO: `/v3/users/dad35ac7d9c847c4bd9792dfe922a3ce` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.228753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad35ac7d9c847c4bd9792dfe922a3ce)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.230748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.248305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:40.283137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad35ac7d9c847c4bd9792dfe922a3ce'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.291033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad35ac7d9c847c4bd9792dfe922a3ce'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.291745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad35ac7d9c847c4bd9792dfe922a3ce'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.301060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad35ac7d9c847c4bd9792dfe922a3ce'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.314021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad35ac7d9c847c4bd9792dfe922a3ce'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.320601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1107/2194] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:40 2022] DELETE /identity/v3/users/dad35ac7d9c847c4bd9792dfe922a3ce => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:40.467935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.470292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.497406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf95289-ba34-4e00-b3cf-aaf5906d2b54 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.627677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.628562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.628847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.629120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] PATH_INFO: `/v3/users/0dafe42edecc4d6a83d94e3805263e51` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.637547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dafe42edecc4d6a83d94e3805263e51)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.639534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.734864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.736834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.749932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dafe42edecc4d6a83d94e3805263e51'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.761677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18572601-5f16-4956-8854-22ea4b1de71d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.764172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dafe42edecc4d6a83d94e3805263e51'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.787668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.788413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.788703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.788991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] PATH_INFO: `/v3/projects/97c56f23aa1445058b9240738ad0e321` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.795177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c56f23aa1445058b9240738ad0e321)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.801218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dafe42edecc4d6a83d94e3805263e51'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.801218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dafe42edecc4d6a83d94e3805263e51'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.802870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.815456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f75385cd-b572-4c62-a3f4-a00391db70b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dafe42edecc4d6a83d94e3805263e51'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.835541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1088/2195] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:40 2022] DELETE /identity/v3/users/0dafe42edecc4d6a83d94e3805263e51 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:40.838981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.839369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.839785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.880919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:40.882874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.916636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c56f23aa1445058b9240738ad0e321', 'request_id': 'req-ef5598f4-6663-43dc-88d3-5724b7cd95c3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.928113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c56f23aa1445058b9240738ad0e321', 'request_id': 'req-ef5598f4-6663-43dc-88d3-5724b7cd95c3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.944161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c56f23aa1445058b9240738ad0e321', 'request_id': 'req-ef5598f4-6663-43dc-88d3-5724b7cd95c3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.965701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] The token cache is being invalidate because project 97c56f23aa1445058b9240738ad0e321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:40.965805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.966694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ef5598f4-6663-43dc-88d3-5724b7cd95c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:40.973363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1108/2196] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:40 2022] DELETE /identity/v3/projects/97c56f23aa1445058b9240738ad0e321 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:40.982421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28361afe-6f46-4cad-abae-2f945136ec84 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.982781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28361afe-6f46-4cad-abae-2f945136ec84 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.983135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28361afe-6f46-4cad-abae-2f945136ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.036922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-28361afe-6f46-4cad-abae-2f945136ec84 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:41.037856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1089/2197] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:41.038185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-28361afe-6f46-4cad-abae-2f945136ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.048497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1109/2198] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:41.056703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.058195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.064337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:41.065432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.065722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.070806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.155159 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:41.156780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.174644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1110/2199] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:41.179853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.195902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26656d1-a1f1-4861-8870-942dfd049dfe None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.201406 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90850c16-0f77-480a-9f75-852b6db2f68f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.207223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.207862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.208706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.208991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.214380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.214997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.215272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.215543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.216308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:41.216514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.218645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.233596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.235064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.262109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1111/2200] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:41.273368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.283622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1090/2201] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:41.317751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821845a-ca53-4d41-97a3-d15494cc9a3c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.332386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.333033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.333368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.333793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.334501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.339730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.404400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1112/2202] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:41 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:41.404594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.421592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbb1f0-3f79-411f-936a-f34073e75876 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.597655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.598794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.599077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.599814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.600797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.603707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.660554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1091/2203] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:41 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:41.665044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.678769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f4cc-2bbd-4aff-86ee-0bb26a767dca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.689349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.690767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.692470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.693573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.696212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.700431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.737265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1113/2204] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:41.739810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.755746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55954-53d9-4963-a3fd-536460a779a8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.769937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.770679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.773814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.774217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.775252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.778209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.791584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1092/2205] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:41.795894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.822221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd5ef1-aeec-4ae5-a92f-388ec86cae47 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.842195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8de820-95fe-4395-8d75-02a4a0e5703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.843445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8de820-95fe-4395-8d75-02a4a0e5703a admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.843959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8de820-95fe-4395-8d75-02a4a0e5703a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.844391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8de820-95fe-4395-8d75-02a4a0e5703a admin admin] PATH_INFO: `/v3/projects/ae8603bb4dd149f48047f0a8fab04e1e/users/20e7edd6a3c04fee99f07970e7dc5b7e/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.856622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.861441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.866047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:41.867652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8de820-95fe-4395-8d75-02a4a0e5703a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8603bb4dd149f48047f0a8fab04e1e, user_id=20e7edd6a3c04fee99f07970e7dc5b7e, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.871567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8de820-95fe-4395-8d75-02a4a0e5703a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.907861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1114/2206] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:41 2022] PUT /identity/v3/projects/ae8603bb4dd149f48047f0a8fab04e1e/users/20e7edd6a3c04fee99f07970e7dc5b7e/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:41.913415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0b1f0-c36d-447a-8e2a-2966ef8601da admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.920118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.920427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.921501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.960503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] MFA Rules not processed for user `20e7edd6a3c04fee99f07970e7dc5b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:41.962258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.015629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.016216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.016492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.016762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] PATH_INFO: `/v3/projects/a7089141e0b44798b98b58d9673176c4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.018011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7089141e0b44798b98b58d9673176c4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.021769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.066516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:42.074794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1115/2207] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:42.080448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.094944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7089141e0b44798b98b58d9673176c4', 'request_id': 'req-becb57ad-d02d-45a2-a46a-1f8ecd98a491'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.096603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6ffa6-b980-445b-96bd-fcda51244f6a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.103833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7089141e0b44798b98b58d9673176c4', 'request_id': 'req-becb57ad-d02d-45a2-a46a-1f8ecd98a491'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.114227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7089141e0b44798b98b58d9673176c4', 'request_id': 'req-becb57ad-d02d-45a2-a46a-1f8ecd98a491'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.133636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] The token cache is being invalidate because project a7089141e0b44798b98b58d9673176c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:42.133939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.134768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-becb57ad-d02d-45a2-a46a-1f8ecd98a491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.136582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1093/2208] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:41 2022] DELETE /identity/v3/projects/a7089141e0b44798b98b58d9673176c4 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:42.141994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c77b97-1436-4210-b2f7-92bd2185179e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.142445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c77b97-1436-4210-b2f7-92bd2185179e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.142859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c77b97-1436-4210-b2f7-92bd2185179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.177674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a9c77b97-1436-4210-b2f7-92bd2185179e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:42.179725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a9c77b97-1436-4210-b2f7-92bd2185179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.191753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.193219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.193617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.194004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.195056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.197863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.272516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1094/2209] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:42.274465 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.274816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.275229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.286922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1116/2210] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:42.291178 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.297255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.342280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:42.345222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.420335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556fac-e7d8-4f0e-b376-bee710649bcd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.425171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.426941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.427283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.427559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.428247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.430230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.446759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:42.454302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1095/2211] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:42.458166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.460927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1117/2212] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:42.464935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.470769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5d21-3f0c-4eda-acc4-b67cf57a70b3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.477605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.478402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.480559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.480559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.480559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:42.480559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.482184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.497975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1096/2213] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:42.499556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732fc9-457c-4d6d-9bb7-36a3d8eff6b6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.503157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.506783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.507496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.507780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.508083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.508786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.511151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.525527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1118/2214] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:42.529523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.542609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54edea6-2906-47e5-bf05-87d8fcae3540 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.546798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3feab1-9315-4a42-99c9-822b91bac29b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.554019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.554234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.554896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.555047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.555355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.555490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.555723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] PATH_INFO: `/v3/projects/36dc93dcc1704566af93e372b85861cd/users/a2faea6044494780a7f6ad4eef5e3e90/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.555883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.557498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.558960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.577443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dc93dcc1704566af93e372b85861cd, user_id=a2faea6044494780a7f6ad4eef5e3e90, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.580313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.598211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1119/2215] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:42 2022] PUT /identity/v3/projects/36dc93dcc1704566af93e372b85861cd/users/a2faea6044494780a7f6ad4eef5e3e90/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:42.602430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.606284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1097/2216] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:42.607391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.673969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:42.780965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195cdad-959e-4abb-acef-409dfe202db0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.789332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.790108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.790398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.790677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.791317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3338cb-7463-43f9-9c63-08e0f3f8135f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.791454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.793421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.796842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.797783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.798233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.798653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.799451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.802196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.819889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1120/2217] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:42.821475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.835194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22134a-6f06-45ce-a940-eac1c9890941 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.843257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950b165-96c3-4319-abb6-c938b486efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.843908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950b165-96c3-4319-abb6-c938b486efd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.844255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950b165-96c3-4319-abb6-c938b486efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.844593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950b165-96c3-4319-abb6-c938b486efd7 admin admin] PATH_INFO: `/v3/projects/36dc93dcc1704566af93e372b85861cd/users/a2faea6044494780a7f6ad4eef5e3e90/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.846941 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950b165-96c3-4319-abb6-c938b486efd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36dc93dcc1704566af93e372b85861cd, user_id=a2faea6044494780a7f6ad4eef5e3e90, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.847971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1098/2218] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:42.851114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950b165-96c3-4319-abb6-c938b486efd7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.853317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.864552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1121/2219] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:42 2022] PUT /identity/v3/projects/36dc93dcc1704566af93e372b85861cd/users/a2faea6044494780a7f6ad4eef5e3e90/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:42.868298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506fa25-552a-4bbd-a213-2a99a3f676d5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.877898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.878187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.879259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.922710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] MFA Rules not processed for user `a2faea6044494780a7f6ad4eef5e3e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:42.923996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.985578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.986182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.986463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.986741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.988600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.990515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.004001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1099/2220] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:43.004001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.005606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.021127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fded47-fc28-46f6-a9e9-71904dbe2337 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:43.032566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:43.032851 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.033129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.033400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] PATH_INFO: `/v3/users/787451aea70c49d6814637d48037202f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.038551 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1122/2221] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:43.039334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=787451aea70c49d6814637d48037202f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:43.039458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.039563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.059486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa3f5-e94e-493a-9a01-bf09707f9fc1 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:43.070194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:43.071391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.071784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.072799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.073906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:43.076289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.094095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1123/2222] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:43.094874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.096736 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787451aea70c49d6814637d48037202f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.099504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.105597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787451aea70c49d6814637d48037202f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.121052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787451aea70c49d6814637d48037202f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.122373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787451aea70c49d6814637d48037202f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.131234 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787451aea70c49d6814637d48037202f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.132748 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:43.142466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1100/2223] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:43 2022] DELETE /identity/v3/users/787451aea70c49d6814637d48037202f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:43.145100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.147460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.179050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:43.186992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a0f67-a840-4766-80fe-ec242e26da49 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:43.250058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774c793-d57a-4aa6-bb8c-98a5ed00f162 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:43.255925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413971a-920a-44f6-8580-1579b838a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:43.256622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413971a-920a-44f6-8580-1579b838a7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.256898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413971a-920a-44f6-8580-1579b838a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.257179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413971a-920a-44f6-8580-1579b838a7fd admin admin] PATH_INFO: `/v3/projects/fe5a43a5024141fc9c9977086b245550/users/ee66b7c264dd44db887898923c925dc8/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.266848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:43.267608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.268057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.268518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.271564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.276946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413971a-920a-44f6-8580-1579b838a7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5a43a5024141fc9c9977086b245550, user_id=ee66b7c264dd44db887898923c925dc8, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:43.280123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413971a-920a-44f6-8580-1579b838a7fd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.301983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1101/2224] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:43 2022] PUT /identity/v3/projects/fe5a43a5024141fc9c9977086b245550/users/ee66b7c264dd44db887898923c925dc8/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:43.310841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.310841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.310841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.347882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:43.349959 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.356736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:43.358822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b0b5e-c051-48fe-8ccf-bad3df6952a8 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.372694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1124/2225] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:43.386443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.389834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.390133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.444194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1102/2226] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:43.458134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] MFA Rules not processed for user `ee66b7c264dd44db887898923c925dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:43.460007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.460883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.463029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.570758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26489a-62e2-4187-bfc7-bffb5e39bd03 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:43.576167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:43.577158 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.583937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.584479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.632069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:43.636508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.644848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1125/2227] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:43.663331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1103/2228] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:43.888991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.891107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.931770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:44.023441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f605e7-d9ec-44ba-901a-c289224e0c88 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:44.032838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee7356-4ec9-4875-82cc-b67c53d3867b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:44.033651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee7356-4ec9-4875-82cc-b67c53d3867b service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:44.033977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee7356-4ec9-4875-82cc-b67c53d3867b service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:44.034248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee7356-4ec9-4875-82cc-b67c53d3867b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:44.051939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee7356-4ec9-4875-82cc-b67c53d3867b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:44.072980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee7356-4ec9-4875-82cc-b67c53d3867b service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:44.094620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1126/2229] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:44.853861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:44.855485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:44.875189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9e420-6aaf-48db-8e3a-e5e0585867a0 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:44.882825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:44.883511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:44.883794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:44.884170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] PATH_INFO: `/v3/projects/cd6a3ffef6b3427893932c409e43994f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:44.885605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6a3ffef6b3427893932c409e43994f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:44.887858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:44.955149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:44.967326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6a3ffef6b3427893932c409e43994f', 'request_id': 'req-6dca1989-39f0-47a5-b830-01715169b020'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:44.996612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6a3ffef6b3427893932c409e43994f', 'request_id': 'req-6dca1989-39f0-47a5-b830-01715169b020'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:45.013583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6a3ffef6b3427893932c409e43994f', 'request_id': 'req-6dca1989-39f0-47a5-b830-01715169b020'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:45.050044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] The token cache is being invalidate because project cd6a3ffef6b3427893932c409e43994f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:45.051405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:45.052235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6dca1989-39f0-47a5-b830-01715169b020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:45.054221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1104/2230] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:44 2022] DELETE /identity/v3/projects/cd6a3ffef6b3427893932c409e43994f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:45.059067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.059500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.059876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.085741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:45.089192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:45.198434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1127/2231] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:45.207119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.207467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.207811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.239805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:45.241321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:45.251295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1105/2232] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:45.255113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:45.266838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6b514-a8ab-44b0-ad2b-593fce364b2c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:45.271996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:45.272851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.273342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.273833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.274709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:45.275226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:45.277102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:45.291878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1128/2233] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:45.296389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:45.307657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317ebd-7a6d-4230-9ef2-d2305ab85b10 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:45.313846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:45.314527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.314816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.315092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.315891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:45.319610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:45.384910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1106/2234] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:45 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:45.386312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:45.406447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2970-6a12-4411-99ff-5425916de940 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:45.520095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:45.521295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.521676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.522047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.523571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:45.527540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:45.576194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1129/2235] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:45 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:45.579432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:45.621506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63efb7a-8360-44c3-a0b3-12570431996e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:45.627910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a69857-5c56-4b5f-a12e-15c5511e29d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:45.629113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a69857-5c56-4b5f-a12e-15c5511e29d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.629657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a69857-5c56-4b5f-a12e-15c5511e29d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.630049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a69857-5c56-4b5f-a12e-15c5511e29d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.631017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a69857-5c56-4b5f-a12e-15c5511e29d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:45.632919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a69857-5c56-4b5f-a12e-15c5511e29d6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:45.654237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1107/2236] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:45.658741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:45.714746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326293-f298-4c98-8569-dacce69be094 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:45.716354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.718652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.719276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.727450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:45.728155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.728554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.739260 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.740476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:45.749790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:45.763712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] MFA Rules not processed for user `ee66b7c264dd44db887898923c925dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:45.768136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:45.809903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1130/2237] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:45.813344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:45.850783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6b54c-ca8d-46df-82be-6f296b93220b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:45.865019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5e883-670e-4bb1-b349-71249f4db6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:45.865964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5e883-670e-4bb1-b349-71249f4db6e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.867050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5e883-670e-4bb1-b349-71249f4db6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.867545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5e883-670e-4bb1-b349-71249f4db6e2 admin admin] PATH_INFO: `/v3/projects/ba9a89eeb01640e291901b3443a017a5/users/9c7a5e92d9c64f1d8f11b800111a25bb/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.887169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5e883-670e-4bb1-b349-71249f4db6e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9a89eeb01640e291901b3443a017a5, user_id=9c7a5e92d9c64f1d8f11b800111a25bb, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:45.890462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e5e883-670e-4bb1-b349-71249f4db6e2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:45.912981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1131/2238] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:45 2022] PUT /identity/v3/projects/ba9a89eeb01640e291901b3443a017a5/users/9c7a5e92d9c64f1d8f11b800111a25bb/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:45.933420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:45.935926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:45.936267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:45.938480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1108/2239] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:45.970464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:46.039337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] MFA Rules not processed for user `9c7a5e92d9c64f1d8f11b800111a25bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:46.040256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:46.193318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1132/2240] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:46.197432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.211892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a703-c7a4-4c72-b1ca-f4bf9019eff5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:46.302018 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:46.303578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.303876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.304180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.304827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:46.307933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:46.366991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1109/2241] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:46 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:46.371170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.389237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54d695-8f3e-48e0-bde4-a5f992c7370e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:46.491890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:46.492560 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.493345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.494239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.494790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:46.497419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:46.546031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1133/2242] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:46 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:46.549928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.560187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.562977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d650a-68da-4aa4-905c-3bac6181076d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:46.565194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:46.576437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:46.577051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.577327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.577595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.583788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:46.585558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:46.598579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819737a-15f5-478a-85e1-f1a72bda0935 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:46.604163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:46.604727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1110/2243] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:46.605368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.605560 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.606021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.606482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] PATH_INFO: `/v3/users/05e175cab43245d7b35df0adc8564394` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.607209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:46.608517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e175cab43245d7b35df0adc8564394)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:46.610651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:46.621801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522e5b-9ce2-4e4f-9e1e-4faef7569c76 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:46.628804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:46.629548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.630771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.631256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] PATH_INFO: `/v3/users/20e7edd6a3c04fee99f07970e7dc5b7e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.632686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20e7edd6a3c04fee99f07970e7dc5b7e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:46.634730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:46.636500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:46.679274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e175cab43245d7b35df0adc8564394'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.688401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e175cab43245d7b35df0adc8564394'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.689425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e175cab43245d7b35df0adc8564394'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.699257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e175cab43245d7b35df0adc8564394'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.702583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20e7edd6a3c04fee99f07970e7dc5b7e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.712836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:46.713402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20e7edd6a3c04fee99f07970e7dc5b7e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.715707 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e175cab43245d7b35df0adc8564394'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.724016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1134/2244] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:46 2022] DELETE /identity/v3/users/05e175cab43245d7b35df0adc8564394 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:46.725230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.727812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20e7edd6a3c04fee99f07970e7dc5b7e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.729241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20e7edd6a3c04fee99f07970e7dc5b7e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.732285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:46.747201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-49fe3977-8521-4991-900f-fc2c47fbcb28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20e7edd6a3c04fee99f07970e7dc5b7e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:46.762055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1111/2245] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:46 2022] DELETE /identity/v3/users/20e7edd6a3c04fee99f07970e7dc5b7e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:46.768674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:46.769685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:46.770191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:46.810827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:46.812072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:46.885099 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb851a48-548a-45d3-bfd0-9945634ef40a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:46.975679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1112/2246] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:46.994527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:46.998837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.129307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.130045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.131984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.132375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.133161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.135408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d733f10-b9c8-490c-9cbf-abc655430932 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.137662 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.139076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.139902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.140141 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.140376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.153719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.155427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.158009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1135/2247] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:47.158913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.164282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.172112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1113/2248] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:47.173320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.174906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.189848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0b5e3-b6cb-4937-9bbb-8a90d5172734 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.197023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dad113-e09f-4fc7-8f99-138257df6e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.197694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dad113-e09f-4fc7-8f99-138257df6e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.197966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dad113-e09f-4fc7-8f99-138257df6e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.198245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dad113-e09f-4fc7-8f99-138257df6e13 admin admin] PATH_INFO: `/v3/projects/76ba6de7dc8a42e5aea309063fa4add9/users/3f839b1f648840b596e63b655ff8c7b1/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.198653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3919-b7c8-40a1-a7d8-c56f89d03e7a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.209301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.210083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.210464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.211494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] PATH_INFO: `/v3/projects/09380cc0f03944468d21fa2571e992d0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.213718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=09380cc0f03944468d21fa2571e992d0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.215770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.220562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dad113-e09f-4fc7-8f99-138257df6e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ba6de7dc8a42e5aea309063fa4add9, user_id=3f839b1f648840b596e63b655ff8c7b1, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.225707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dad113-e09f-4fc7-8f99-138257df6e13 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.248402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:47.259003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1114/2249] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:47 2022] PUT /identity/v3/projects/76ba6de7dc8a42e5aea309063fa4add9/users/3f839b1f648840b596e63b655ff8c7b1/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:47.270981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.271249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.271513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.312850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] MFA Rules not processed for user `3f839b1f648840b596e63b655ff8c7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:47.314674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.316877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09380cc0f03944468d21fa2571e992d0', 'request_id': 'req-61fbd18d-6c9f-4377-8a48-00c19b7412eb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:47.325349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09380cc0f03944468d21fa2571e992d0', 'request_id': 'req-61fbd18d-6c9f-4377-8a48-00c19b7412eb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:47.348484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09380cc0f03944468d21fa2571e992d0', 'request_id': 'req-61fbd18d-6c9f-4377-8a48-00c19b7412eb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:47.365225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] The token cache is being invalidate because project 09380cc0f03944468d21fa2571e992d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:47.365831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:47.367203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61fbd18d-6c9f-4377-8a48-00c19b7412eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:47.369005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1136/2250] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:47 2022] DELETE /identity/v3/projects/09380cc0f03944468d21fa2571e992d0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:47.376936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cf58-dd4a-4d7b-b651-19bd18f6fa16 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.377946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cf58-dd4a-4d7b-b651-19bd18f6fa16 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.378946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cf58-dd4a-4d7b-b651-19bd18f6fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.405132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e230cf58-dd4a-4d7b-b651-19bd18f6fa16 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:47.407292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e230cf58-dd4a-4d7b-b651-19bd18f6fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.451253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:47.467816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1115/2251] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:47.475884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.478250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.584663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1137/2252] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:47.592177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.592732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.593475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.631734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:47.632987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.640561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969ed61-f7b9-40b7-81dc-789408050b63 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.652434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1138/2253] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:47.653015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.665212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.666038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.666573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.666952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.667699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.670094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.672198 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279bc45-e6a4-406e-88ef-c7e0de6f5485 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.682060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.682688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.682951 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.683221 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.684035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:47.684337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.688503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.705988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1139/2254] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:47.710963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.719534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1116/2255] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:47.723853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.740078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b06f2e-c7b2-471e-8f48-61564e748584 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.816442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c245-3614-42ef-9085-3a95ec0c511c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.831897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.832579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.832855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.833113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.836290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.837195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.837817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.838096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.838373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.839007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.839271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.841313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.866945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1117/2256] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:47.868008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.870011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:47.875431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1140/2257] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:47.876383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.882825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:47.888143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc24c8-4edf-428e-b0f1-95803f42ef37 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.958853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba2c79-dbcc-4331-8f3d-8e31ea429d31 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:47.966753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.967496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.967774 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.968911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.969660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.972935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.983855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1141/2258] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:47.984931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:47.987752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:47.989173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:47.989460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:47.989737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] PATH_INFO: `/v3/projects/ae8603bb4dd149f48047f0a8fab04e1e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:47.990892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae8603bb4dd149f48047f0a8fab04e1e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:47.994567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:47.996625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd220c4-7cb8-4106-a960-7fba30a87c83 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.012772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.013463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.013745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.014227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.015015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.017752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.056289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1142/2259] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:47 2022] POST /identity/v3/users => generated 293 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:48.057632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.065542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:48.071587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829472ba-11c0-41f4-bc26-4b651a4d68f5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.084092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae8603bb4dd149f48047f0a8fab04e1e', 'request_id': 'req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.092752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae8603bb4dd149f48047f0a8fab04e1e', 'request_id': 'req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.100957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae8603bb4dd149f48047f0a8fab04e1e', 'request_id': 'req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.121146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] The token cache is being invalidate because project ae8603bb4dd149f48047f0a8fab04e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:48.121288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.122392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.125322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1118/2260] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:47 2022] DELETE /identity/v3/projects/ae8603bb4dd149f48047f0a8fab04e1e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:48.126132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.135189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.154679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.155665 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.156181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.156599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] PATH_INFO: `/v3/projects/377caaf39dcb4bf88c149bc1f18beaff/users/c90ff6b2ae6d42b6b39da51433eed8c6/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.176908 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385eeb-0a82-44a2-afdf-5900b8a7fe4d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.188740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.189345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.190095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.190095 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.190546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.192555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.193800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=377caaf39dcb4bf88c149bc1f18beaff, user_id=c90ff6b2ae6d42b6b39da51433eed8c6, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.200613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.219305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1119/2261] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:48.224489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1143/2262] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:48 2022] PUT /identity/v3/projects/377caaf39dcb4bf88c149bc1f18beaff/users/c90ff6b2ae6d42b6b39da51433eed8c6/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:48.225452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.225564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.228568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.246546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea509c1-1ae7-4b7d-8464-b646bea7508f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.311833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91459727-4467-4d25-a25c-b6f3fa7a3c0f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.316612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.317214 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.317489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.317765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.318423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.320033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.323997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.324695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.325048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.325390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] PATH_INFO: `/v3/users/a2faea6044494780a7f6ad4eef5e3e90` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.326637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2faea6044494780a7f6ad4eef5e3e90)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.328636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.334046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1144/2263] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:48.334046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.334046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.346668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02375c59-17ea-4cb9-ac6a-4c033727d373 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.352556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.353166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.353431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.353702 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.354420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.355509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:48.356195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.368357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1145/2264] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:48.369288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.379756 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18a88c-aca0-474d-a831-70fd171158cf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.384441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2faea6044494780a7f6ad4eef5e3e90'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.386236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e28e1c-6dac-4844-b01a-8f9c8a1a070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.387018 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28e1c-6dac-4844-b01a-8f9c8a1a070c admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.387305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28e1c-6dac-4844-b01a-8f9c8a1a070c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.387583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28e1c-6dac-4844-b01a-8f9c8a1a070c admin admin] PATH_INFO: `/v3/projects/2a422dfab790479db363b57365d16320/users/3b44aa2ad6c7432fa333613cbd5bfc96/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.392624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2faea6044494780a7f6ad4eef5e3e90'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.404988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e28e1c-6dac-4844-b01a-8f9c8a1a070c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a422dfab790479db363b57365d16320, user_id=3b44aa2ad6c7432fa333613cbd5bfc96, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.406125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2faea6044494780a7f6ad4eef5e3e90'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.406809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2faea6044494780a7f6ad4eef5e3e90'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.407675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e28e1c-6dac-4844-b01a-8f9c8a1a070c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.414757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2faea6044494780a7f6ad4eef5e3e90'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:48.421024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1120/2265] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:48 2022] DELETE /identity/v3/users/a2faea6044494780a7f6ad4eef5e3e90 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:48.424203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.424327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1146/2266] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:48 2022] PUT /identity/v3/projects/2a422dfab790479db363b57365d16320/users/3b44aa2ad6c7432fa333613cbd5bfc96/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:48.424432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.433023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.433391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.433858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.469170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] MFA Rules not processed for user `3b44aa2ad6c7432fa333613cbd5bfc96`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:48.470345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.516447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56070b2-9c2f-42d9-ba5a-c80572a72d42 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.521077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12910d8-ab79-428f-8445-9d840ea6d4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.521751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12910d8-ab79-428f-8445-9d840ea6d4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.522033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12910d8-ab79-428f-8445-9d840ea6d4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.522312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12910d8-ab79-428f-8445-9d840ea6d4eb admin admin] PATH_INFO: `/v3/projects/377caaf39dcb4bf88c149bc1f18beaff/users/c90ff6b2ae6d42b6b39da51433eed8c6/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.523589 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:48.524435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12910d8-ab79-428f-8445-9d840ea6d4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=377caaf39dcb4bf88c149bc1f18beaff, user_id=c90ff6b2ae6d42b6b39da51433eed8c6, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:48.535553 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12910d8-ab79-428f-8445-9d840ea6d4eb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:48.553794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1121/2267] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:48 2022] PUT /identity/v3/projects/377caaf39dcb4bf88c149bc1f18beaff/users/c90ff6b2ae6d42b6b39da51433eed8c6/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:48.565496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.566600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.567085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1147/2268] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:48.567255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.578578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.580822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.605529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] MFA Rules not processed for user `c90ff6b2ae6d42b6b39da51433eed8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:48.606753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.765764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215ad7d-a7d1-468f-8de0-eb55448bb98a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.778503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1122/2269] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:48.787425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:48.789904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:48.833805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:48.902614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.904394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.905864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.907624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.921937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953a908-8c59-4d1c-aabb-a6a3a4872f99 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:48.927513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:48.928279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:48.928651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:48.929212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:48.932436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:49.005574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:49.008204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:49.039647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1148/2270] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:49.084001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:49.084001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:49.104071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:49.115050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:49.116654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b20ca-989c-4754-b410-9117fb5b2936 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:49.127092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1123/2271] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:40:49.244170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b619a28-a79b-4fc7-abc2-0817f6fcee4d service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:49.257094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:49.258276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.258767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.259369 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] PATH_INFO: `/v3/projects/36dc93dcc1704566af93e372b85861cd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.264620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36dc93dcc1704566af93e372b85861cd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:49.266767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:49.396123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36dc93dcc1704566af93e372b85861cd', 'request_id': 'req-08560f55-0e79-4455-aa78-5d6079fda512'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:49.406234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36dc93dcc1704566af93e372b85861cd', 'request_id': 'req-08560f55-0e79-4455-aa78-5d6079fda512'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:49.422544 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36dc93dcc1704566af93e372b85861cd', 'request_id': 'req-08560f55-0e79-4455-aa78-5d6079fda512'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:49.447682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] The token cache is being invalidate because project 36dc93dcc1704566af93e372b85861cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:49.449590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:49.451769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-08560f55-0e79-4455-aa78-5d6079fda512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:49.456067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1149/2272] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:49 2022] DELETE /identity/v3/projects/36dc93dcc1704566af93e372b85861cd => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:49.460199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.460726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.461160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.516257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:49.520569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:49.633770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1124/2273] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:49.636613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.636954 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.637302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.704048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:49.704048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:49.719251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1150/2274] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:49.720940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:49.733259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89cef7-7d1c-419d-9c11-24704989596d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:49.747261 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:49.747932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.748249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.748520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.749172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:49.749473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:49.751910 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:49.779408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1125/2275] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:49.782002 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:49.794204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c68d-afcb-4d4e-bb68-f873eab4f42b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:49.809317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:49.809550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.809821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.810094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.811976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:49.820095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:49.878592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1151/2276] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:49 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:49.883125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:49.897537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa3528-f8a9-4e46-bdc7-860118cb84e3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:49.988410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:49.988735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:49.989016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:49.989286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:49.989960 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:49.992010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:50.005781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:50.024065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1126/2277] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:49 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:50.028993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:50.041791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694ec83-e57f-4b85-bc48-cfef0a66b751 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:50.048754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:50.049686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.051735 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.052077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.052762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:50.054786 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:50.069351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1152/2278] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:50.070543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:50.081109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccabce-f582-48f7-a911-9438d91827c7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:50.091460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21e2e8-5508-46b2-9cdc-25b12b582586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:50.092109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e2e8-5508-46b2-9cdc-25b12b582586 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.092394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e2e8-5508-46b2-9cdc-25b12b582586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.092665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e2e8-5508-46b2-9cdc-25b12b582586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.093357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21e2e8-5508-46b2-9cdc-25b12b582586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:50.095605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21e2e8-5508-46b2-9cdc-25b12b582586 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:50.107991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1127/2279] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:40:50.116013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:50.127288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a1b80-814e-406d-adc4-4da4d01a2848 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:50.133433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:50.134285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.134695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.134978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] PATH_INFO: `/v3/projects/79eb9f1de490416b81901b0421ef275c/users/79aa10274b604aa4af14fbe1b404cfce/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.152159 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79eb9f1de490416b81901b0421ef275c, user_id=79aa10274b604aa4af14fbe1b404cfce, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:50.155097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:50.170631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1153/2280] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:50 2022] PUT /identity/v3/projects/79eb9f1de490416b81901b0421ef275c/users/79aa10274b604aa4af14fbe1b404cfce/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:50.181944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.182495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.182495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.216175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] MFA Rules not processed for user `79aa10274b604aa4af14fbe1b404cfce`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:50.218242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:50.325504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1128/2281] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:50.340737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:50.343939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:50.420831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:50.493339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6da9b-14ff-4a71-8490-53ca3b6a33e5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:50.498143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:50.499245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:50.499595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:50.500220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:50.513129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:50.514834 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:50.543511 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1154/2282] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:51.175883 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:51.178997 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:51.301281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e06c82-1815-4deb-b06a-181248af5d31 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:51.306194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:51.307799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:51.308321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:51.312783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:51.316730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:51.404081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:51.405608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:51.424068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1129/2283] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:52.785850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:52.794889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd589e19-250b-4d3c-b5ba-887b95e4bb57 service glance] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:52.798540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:52.799127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:52.799408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:52.799686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:52.802107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:52.863799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:52.865242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:52.874565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1155/2284] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:57.017072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:57.019195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:57.062990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:57.118578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9166bb-d01c-4738-bf8b-e7ca3a7522f7 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:57.124061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:57.124971 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:57.125147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:57.125431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] PATH_INFO: `/v3/users/412399f7b74b48d685f8ba7b305ad6a8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:57.126676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=412399f7b74b48d685f8ba7b305ad6a8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:57.129893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:57.202399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412399f7b74b48d685f8ba7b305ad6a8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.211747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:57.213083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:57.225774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412399f7b74b48d685f8ba7b305ad6a8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.228829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc098f4-6898-4176-b9ca-e3f54a1ddad8 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:57.234131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:57.234769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:57.235048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:57.235323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] PATH_INFO: `/v3/users/3b44aa2ad6c7432fa333613cbd5bfc96` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:57.236484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b44aa2ad6c7432fa333613cbd5bfc96)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:57.238818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:57.247816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412399f7b74b48d685f8ba7b305ad6a8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.248734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412399f7b74b48d685f8ba7b305ad6a8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.259237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fc5f4294-a2e2-45ff-9067-f68fe3ab5de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412399f7b74b48d685f8ba7b305ad6a8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.267393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1130/2285] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:57 2022] DELETE /identity/v3/users/412399f7b74b48d685f8ba7b305ad6a8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:57.317325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b44aa2ad6c7432fa333613cbd5bfc96'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.324888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b44aa2ad6c7432fa333613cbd5bfc96'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.325856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b44aa2ad6c7432fa333613cbd5bfc96'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.334287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b44aa2ad6c7432fa333613cbd5bfc96'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.348262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b44aa2ad6c7432fa333613cbd5bfc96'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.354876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1156/2286] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:57 2022] DELETE /identity/v3/users/3b44aa2ad6c7432fa333613cbd5bfc96 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:57.358965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:57.359337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:57.359676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:57.381291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:57.382646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:57.448437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1131/2287] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:40:57.458353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:57.459772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:57.480329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:57.481805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:57.499073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6e97-4c47-4112-9928-3997c3023467 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:57.504718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:57.505419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:57.505703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:57.505981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] PATH_INFO: `/v3/projects/a999efed32524f878ccbb21215d3f3cd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:57.507164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a999efed32524f878ccbb21215d3f3cd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:57.509120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:57.510504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:57.541856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717854d-9c02-44db-b42c-2d37a3f8132f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:57.546317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:57.546950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:57.547234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:57.547507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:57.556632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:57.559317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:57.561388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:57.564167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a999efed32524f878ccbb21215d3f3cd', 'request_id': 'req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.571636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a999efed32524f878ccbb21215d3f3cd', 'request_id': 'req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.573439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1157/2288] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:57.577871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a999efed32524f878ccbb21215d3f3cd', 'request_id': 'req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.590753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] The token cache is being invalidate because project a999efed32524f878ccbb21215d3f3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:57.591451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.592177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.594033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1132/2289] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:57 2022] DELETE /identity/v3/projects/a999efed32524f878ccbb21215d3f3cd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:57.598468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:57.608242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71590267-f646-43e8-8c33-8d9e4c89e95a service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:57.673425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:57.674068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:57.674344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:57.675126 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] PATH_INFO: `/v3/users/1b075b4f67f345afa7d42b336815f6c1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:57.676515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b075b4f67f345afa7d42b336815f6c1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:57.678508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:57.740436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b075b4f67f345afa7d42b336815f6c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.748426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b075b4f67f345afa7d42b336815f6c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.749078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b075b4f67f345afa7d42b336815f6c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.762203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b075b4f67f345afa7d42b336815f6c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.802292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b075b4f67f345afa7d42b336815f6c1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:57.802292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1158/2290] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:57 2022] DELETE /identity/v3/users/1b075b4f67f345afa7d42b336815f6c1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:57.920121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:57.921686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:57.936427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81317ee-e87b-4cce-ac37-a6f20e1d14ab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:57.995681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.004516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.018376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.019784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.020101 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.020444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] PATH_INFO: `/v3/users/9c7a5e92d9c64f1d8f11b800111a25bb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.021629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c7a5e92d9c64f1d8f11b800111a25bb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.028931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.085968 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:58.110735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c7a5e92d9c64f1d8f11b800111a25bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.124932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c7a5e92d9c64f1d8f11b800111a25bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.137125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a3bb-3309-4fce-b191-acba923f5f94 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.142685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c7a5e92d9c64f1d8f11b800111a25bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.144148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c7a5e92d9c64f1d8f11b800111a25bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.147021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.147576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.148026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.148312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] PATH_INFO: `/v3/projects/2a422dfab790479db363b57365d16320` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.150118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a422dfab790479db363b57365d16320)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.152496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.155362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c7a5e92d9c64f1d8f11b800111a25bb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.163147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1133/2291] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:57 2022] DELETE /identity/v3/users/9c7a5e92d9c64f1d8f11b800111a25bb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:58.164329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.166115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.222434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a422dfab790479db363b57365d16320', 'request_id': 'req-f801a17e-9f99-4351-9131-a4f1528a564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.230135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a422dfab790479db363b57365d16320', 'request_id': 'req-f801a17e-9f99-4351-9131-a4f1528a564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.235270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:58.241785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a422dfab790479db363b57365d16320', 'request_id': 'req-f801a17e-9f99-4351-9131-a4f1528a564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.261453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] The token cache is being invalidate because project 2a422dfab790479db363b57365d16320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:58.262116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.263013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f801a17e-9f99-4351-9131-a4f1528a564b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.268078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1159/2292] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:57 2022] DELETE /identity/v3/projects/2a422dfab790479db363b57365d16320 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:40:58.272555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45585a0-e046-4a13-a42d-52371c167053 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.273770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab6425-8e4c-496f-9a09-9061d7438321 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.274052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab6425-8e4c-496f-9a09-9061d7438321 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.274327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab6425-8e4c-496f-9a09-9061d7438321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.302910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-29ab6425-8e4c-496f-9a09-9061d7438321 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:58.304202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-29ab6425-8e4c-496f-9a09-9061d7438321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.370807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.371451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.372334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.372555 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] PATH_INFO: `/v3/projects/a4b039bb36fd4a44968e5f71a8dea2d3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.376390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b039bb36fd4a44968e5f71a8dea2d3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.381022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.387076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1160/2293] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:58.388470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6a14b-74a0-4dd8-bcc3-d1688aa45109 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.388759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6a14b-74a0-4dd8-bcc3-d1688aa45109 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.389037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6a14b-74a0-4dd8-bcc3-d1688aa45109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.462366 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-39b6a14b-74a0-4dd8-bcc3-d1688aa45109 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:58.463513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b039bb36fd4a44968e5f71a8dea2d3', 'request_id': 'req-39faf605-74d3-4a8c-b7dd-16632ff926d5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.463657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-39b6a14b-74a0-4dd8-bcc3-d1688aa45109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.472440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b039bb36fd4a44968e5f71a8dea2d3', 'request_id': 'req-39faf605-74d3-4a8c-b7dd-16632ff926d5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.481767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b039bb36fd4a44968e5f71a8dea2d3', 'request_id': 'req-39faf605-74d3-4a8c-b7dd-16632ff926d5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.500361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] The token cache is being invalidate because project a4b039bb36fd4a44968e5f71a8dea2d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:58.500893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.501644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:58.504741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1134/2294] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:58 2022] DELETE /identity/v3/projects/a4b039bb36fd4a44968e5f71a8dea2d3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:58.508655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.511175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.556132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1161/2295] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:58.556132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fb0b-79aa-4d1f-ac04-a726702d9d75 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.556132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fb0b-79aa-4d1f-ac04-a726702d9d75 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.556132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fb0b-79aa-4d1f-ac04-a726702d9d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.581661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a1a4fb0b-79aa-4d1f-ac04-a726702d9d75 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:58.583048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a1a4fb0b-79aa-4d1f-ac04-a726702d9d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.585466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faf605-74d3-4a8c-b7dd-16632ff926d5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.589468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.590087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.590355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.590627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.592245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1162/2296] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:58.592680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.593606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.593890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.594167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.609293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.610750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56c401-dc3a-4338-987c-22efbb1f7019 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.617675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:58.618914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.621708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1135/2297] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:58.622761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.623202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.623498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.628447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1163/2298] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:58.632614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.647668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:58.649673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:58.651637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc65e-b01a-45a1-80fb-f9f5e6418a68 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.663162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.663939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.664253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.664632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.665411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:58.665908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.667708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.669252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1136/2299] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:58.670737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:58.673930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.684728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1164/2300] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:40:58.688870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.692028 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee82200-8372-4cf7-85b7-fc5a0486793f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.699667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.700303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.700644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.700949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.701404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a738-d359-4306-87fc-76f5ad706284 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.701977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:58.702417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.704238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.707935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.708656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.709008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.709802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.711134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.713435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.723562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1137/2301] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:40:58.725737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.747167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8f0f8-95ca-4bd7-9160-f6db79b39ce2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.755671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.756692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.757170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.757975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.759000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.762372 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.768359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:58.772462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1165/2302] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:58.778548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.824339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1138/2303] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:58.828389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:58.900867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68deec7-2621-4dae-be31-1b3346c38d19 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.936126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b97305-17bb-43d2-a299-e4759463fec9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:58.943347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.944740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.945134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.945629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.946322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.951226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:58.978720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:58.979335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:58.979618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:58.980059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:58.980696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:58.982608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.013764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1139/2304] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:40:59.015837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.020828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.038576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a609-af9d-470a-9327-f5c3e223fc11 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.043937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.044683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.048301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.048459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1166/2305] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:58 2022] POST /identity/v3/users => generated 294 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:59.048548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] PATH_INFO: `/v3/projects/ba9a89eeb01640e291901b3443a017a5` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.048623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.048711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9a89eeb01640e291901b3443a017a5)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.049406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.057448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1092f26-c24c-41a2-b87a-566d08ba68e1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.063193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.063825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.064169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.064401 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.065046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.066842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.079463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1167/2306] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:59.080455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.092824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7306422-411f-4cbd-b53a-db8d3768020d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.098639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.099542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.099542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.099802 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.100438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.102014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.114765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1168/2307] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:59.115697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.122501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9a89eeb01640e291901b3443a017a5', 'request_id': 'req-385ddb04-30d6-4014-b07b-44ff5ded914c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.125818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30d77a-3c75-42b9-8afc-5474685cc9ed admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.130574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9a89eeb01640e291901b3443a017a5', 'request_id': 'req-385ddb04-30d6-4014-b07b-44ff5ded914c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.142691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9a89eeb01640e291901b3443a017a5', 'request_id': 'req-385ddb04-30d6-4014-b07b-44ff5ded914c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.159636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] The token cache is being invalidate because project ba9a89eeb01640e291901b3443a017a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:59.160285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.161072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.162932 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1140/2308] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:59 2022] DELETE /identity/v3/projects/ba9a89eeb01640e291901b3443a017a5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:59.163631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.165127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.178245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.199407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.200078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.200350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.200623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.201273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.202882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.215784 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1169/2309] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:59.216500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.219527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.233998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de53f85-0381-4625-a9fd-40b4f511bd0c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.240084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.240721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.241009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.245606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] PATH_INFO: `/v3/users/3f839b1f648840b596e63b655ff8c7b1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.247193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f839b1f648840b596e63b655ff8c7b1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.250267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.252436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.259216 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ddb04-30d6-4014-b07b-44ff5ded914c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.266860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.268089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.268198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.268649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.269831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.272040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.286823 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1141/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:59.290445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.291586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.304822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f839b1f648840b596e63b655ff8c7b1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.305163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bf689-4d71-4156-8ed3-88ddf6888f70 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.309918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce710e-f230-4932-9e18-a7ff8d747ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.310798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce710e-f230-4932-9e18-a7ff8d747ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.311114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce710e-f230-4932-9e18-a7ff8d747ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.311366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce710e-f230-4932-9e18-a7ff8d747ea2 admin admin] PATH_INFO: `/v3/projects/3b6f310cbbc543bc920a27277677b7b3/users/6671429166f54cbcba7a2ac9a525a024/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.313553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f839b1f648840b596e63b655ff8c7b1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.314907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f839b1f648840b596e63b655ff8c7b1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.329780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f839b1f648840b596e63b655ff8c7b1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.337081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce710e-f230-4932-9e18-a7ff8d747ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6f310cbbc543bc920a27277677b7b3, user_id=6671429166f54cbcba7a2ac9a525a024, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.340182 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce710e-f230-4932-9e18-a7ff8d747ea2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.346420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f839b1f648840b596e63b655ff8c7b1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:59.358533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1170/2311] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:59 2022] DELETE /identity/v3/users/3f839b1f648840b596e63b655ff8c7b1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:59.359777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.363994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1142/2312] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:59 2022] PUT /identity/v3/projects/3b6f310cbbc543bc920a27277677b7b3/users/6671429166f54cbcba7a2ac9a525a024/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:59.371540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.373278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.373770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.374245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.421485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] MFA Rules not processed for user `6671429166f54cbcba7a2ac9a525a024`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:59.422944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.478782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d509b2-1bff-4bc6-8a57-e82ae7ec5c58 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.489804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f048c60-1c34-4027-b672-fd7449cc4d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.490903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f048c60-1c34-4027-b672-fd7449cc4d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.491256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f048c60-1c34-4027-b672-fd7449cc4d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.491614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f048c60-1c34-4027-b672-fd7449cc4d62 admin admin] PATH_INFO: `/v3/projects/f213d7be9c1c4a0d8d2e4e8b4950deb6/users/0cc25c9697d64e28970709fc6c0f35a4/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.510189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f048c60-1c34-4027-b672-fd7449cc4d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f213d7be9c1c4a0d8d2e4e8b4950deb6, user_id=0cc25c9697d64e28970709fc6c0f35a4, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.517240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1143/2313] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:59.517447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f048c60-1c34-4027-b672-fd7449cc4d62 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.535074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.537832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.543463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1171/2314] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:59 2022] PUT /identity/v3/projects/f213d7be9c1c4a0d8d2e4e8b4950deb6/users/0cc25c9697d64e28970709fc6c0f35a4/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:59.554977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.555612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.555887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.603329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] MFA Rules not processed for user `0cc25c9697d64e28970709fc6c0f35a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:59.605481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:59.656788 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d909af-3d12-4eee-b4fd-fd1181cf4792 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.668797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.722519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1172/2315] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:40:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:59.727302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.746390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.747026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.747307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.748684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.805701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:59.881352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf453a-b3f3-4379-90ca-3a993d537fc4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.886844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:59.887706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:59.888011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:59.888546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:59.889301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.892035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.899490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:59.902055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:59.927695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1144/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:59.941753 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1173/2317] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:59 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:59.945924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.960574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:59.964264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad0083-1966-42a6-bb2d-0d90551a972e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:59.966901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.067102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.067897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.068052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.068324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.068921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.070909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.082503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72929020-412c-4c6a-8262-fb8e1caeca18 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.088436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.089120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.089393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.089667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] PATH_INFO: `/v3/projects/76ba6de7dc8a42e5aea309063fa4add9` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.090797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ba6de7dc8a42e5aea309063fa4add9)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.092806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.100976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1145/2318] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:59 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:00.100976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.100976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.112415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e5f0d-805e-4435-97b0-aca682d0c25e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.117049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.117675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.117954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.118226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] PATH_INFO: `/v3/users/6671429166f54cbcba7a2ac9a525a024` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.119506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6671429166f54cbcba7a2ac9a525a024)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.121457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.145903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ba6de7dc8a42e5aea309063fa4add9', 'request_id': 'req-1c18086c-4e6b-41a3-8bd4-5731970b1c13'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.154982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ba6de7dc8a42e5aea309063fa4add9', 'request_id': 'req-1c18086c-4e6b-41a3-8bd4-5731970b1c13'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.157850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:00.164276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ba6de7dc8a42e5aea309063fa4add9', 'request_id': 'req-1c18086c-4e6b-41a3-8bd4-5731970b1c13'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.176956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] The token cache is being invalidate because project 76ba6de7dc8a42e5aea309063fa4add9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:00.177245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.177642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6671429166f54cbcba7a2ac9a525a024'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.177897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.179408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1174/2319] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:59 2022] DELETE /identity/v3/projects/76ba6de7dc8a42e5aea309063fa4add9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:00.180353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.181848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.185283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6671429166f54cbcba7a2ac9a525a024'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.198567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6671429166f54cbcba7a2ac9a525a024'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.199255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6671429166f54cbcba7a2ac9a525a024'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.207537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6671429166f54cbcba7a2ac9a525a024'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.213925 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1146/2320] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:00 2022] DELETE /identity/v3/users/6671429166f54cbcba7a2ac9a525a024 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:00.214741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.216142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.252615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:00.254616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18086c-4e6b-41a3-8bd4-5731970b1c13 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.289265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74870a0d-3c5a-420a-b21e-9ed38674bc01 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.294301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.295254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.295725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.296237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] PATH_INFO: `/v3/users/c90ff6b2ae6d42b6b39da51433eed8c6` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.297477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90ff6b2ae6d42b6b39da51433eed8c6)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.299419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.320075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d6d22-94c2-4833-be1f-85cd7bf61438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.320684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d6d22-94c2-4833-be1f-85cd7bf61438 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.320959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d6d22-94c2-4833-be1f-85cd7bf61438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.321231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d6d22-94c2-4833-be1f-85cd7bf61438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.322001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d6d22-94c2-4833-be1f-85cd7bf61438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.323585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d6d22-94c2-4833-be1f-85cd7bf61438 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.334781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1175/2321] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:00.336146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.336477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.336762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.348742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90ff6b2ae6d42b6b39da51433eed8c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.355380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:00.356601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90ff6b2ae6d42b6b39da51433eed8c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.356842 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.365218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1176/2322] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:00.372812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.372812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.376582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90ff6b2ae6d42b6b39da51433eed8c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.377755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90ff6b2ae6d42b6b39da51433eed8c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.384816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b659e5-33ce-4b6d-835d-c786934347b0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.387293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90ff6b2ae6d42b6b39da51433eed8c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:00.389855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.390493 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.390837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.391056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.391705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.393376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1147/2323] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:00 2022] DELETE /identity/v3/users/c90ff6b2ae6d42b6b39da51433eed8c6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:00.393484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.394310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.396082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.409461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1177/2324] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:00.410689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.412621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.427236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012377-d258-40b3-98be-a084bb61e08e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.481097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb66d80-4116-4c68-ae5f-a422b5b53163 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.489040 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.489933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.490535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.490733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.492695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.520352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b82888-b03e-49d7-8a02-47bcd3a30ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.521053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b82888-b03e-49d7-8a02-47bcd3a30ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.521398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b82888-b03e-49d7-8a02-47bcd3a30ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.521751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b82888-b03e-49d7-8a02-47bcd3a30ff4 admin admin] PATH_INFO: `/v3/projects/0cbe66ffbacb40f0b9dfada5461cec9b/users/7df85dad4f054ed2ac6e7708e2c63de7/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.539328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b82888-b03e-49d7-8a02-47bcd3a30ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbe66ffbacb40f0b9dfada5461cec9b, user_id=7df85dad4f054ed2ac6e7708e2c63de7, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.542760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b82888-b03e-49d7-8a02-47bcd3a30ff4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.545364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:00.572900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1178/2325] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:00 2022] PUT /identity/v3/projects/0cbe66ffbacb40f0b9dfada5461cec9b/users/7df85dad4f054ed2ac6e7708e2c63de7/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:00.589428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.590083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.590504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.607570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.609361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.616499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:00.623235 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1148/2326] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:00.631477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] MFA Rules not processed for user `7df85dad4f054ed2ac6e7708e2c63de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:00.633368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.740313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1179/2327] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:00.744714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.757318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129593d-7d11-4fee-8606-c5a54982c870 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:00.839544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:00.851587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:00.852175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:00.852554 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:00.853522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:00.856061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:00.918878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.919516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1149/2328] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:00 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:00.920438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:00.923316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:00.942197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea243-5bf7-4c8b-bd5d-2c49e72cff6b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.004952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33009e5-1fc4-4a22-91b8-80912bb53fc4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.019667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.020887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.021322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.021835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.022581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.025408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.068977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1150/2329] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:00 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:01.068977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.069696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.069912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.070001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.070001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.070165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] PATH_INFO: `/v3/projects/377caaf39dcb4bf88c149bc1f18beaff` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.071365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] RBAC: Authorizing `identity:delete_project(project_id=377caaf39dcb4bf88c149bc1f18beaff)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.073553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.084833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe287-49e5-4e77-84f3-1824eae18fbb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.117236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.117798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.118080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.118356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] PATH_INFO: `/v3/projects/3b6f310cbbc543bc920a27277677b7b3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.124311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6f310cbbc543bc920a27277677b7b3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.126568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.175491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377caaf39dcb4bf88c149bc1f18beaff', 'request_id': 'req-7ec88572-9e38-40cd-a442-0d9c5353435c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.183087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377caaf39dcb4bf88c149bc1f18beaff', 'request_id': 'req-7ec88572-9e38-40cd-a442-0d9c5353435c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.191881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377caaf39dcb4bf88c149bc1f18beaff', 'request_id': 'req-7ec88572-9e38-40cd-a442-0d9c5353435c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.199217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6f310cbbc543bc920a27277677b7b3', 'request_id': 'req-db4962aa-2832-4332-aa54-8312c5398d2e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.202995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] The token cache is being invalidate because project 377caaf39dcb4bf88c149bc1f18beaff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:01.203349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.204067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.206071 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1180/2330] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:00 2022] DELETE /identity/v3/projects/377caaf39dcb4bf88c149bc1f18beaff => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:01.207087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.208469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.208789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6f310cbbc543bc920a27277677b7b3', 'request_id': 'req-db4962aa-2832-4332-aa54-8312c5398d2e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.215354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6f310cbbc543bc920a27277677b7b3', 'request_id': 'req-db4962aa-2832-4332-aa54-8312c5398d2e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.222186 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:01.225657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:01.234298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] The token cache is being invalidate because project 3b6f310cbbc543bc920a27277677b7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:01.234298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.234688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-db4962aa-2832-4332-aa54-8312c5398d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:01.237773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1151/2331] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:01 2022] DELETE /identity/v3/projects/3b6f310cbbc543bc920a27277677b7b3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:01.240147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366c58-8ead-480c-94f6-b5dd95c2e4c0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.240436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366c58-8ead-480c-94f6-b5dd95c2e4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.240707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366c58-8ead-480c-94f6-b5dd95c2e4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.261795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a7366c58-8ead-480c-94f6-b5dd95c2e4c0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:01.263007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a7366c58-8ead-480c-94f6-b5dd95c2e4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.309104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec88572-9e38-40cd-a442-0d9c5353435c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.334045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1152/2332] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:01.335491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02ace-7753-4dae-b5ba-220506dccc09 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.335771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02ace-7753-4dae-b5ba-220506dccc09 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.336098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02ace-7753-4dae-b5ba-220506dccc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.362672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-91c02ace-7753-4dae-b5ba-220506dccc09 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:01.362672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-91c02ace-7753-4dae-b5ba-220506dccc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.370744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1153/2333] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:01.372245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0398c2-e245-475a-9611-ae4c30dd9e0b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.372526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0398c2-e245-475a-9611-ae4c30dd9e0b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.372799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0398c2-e245-475a-9611-ae4c30dd9e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.395929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-be0398c2-e245-475a-9611-ae4c30dd9e0b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:01.397068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-be0398c2-e245-475a-9611-ae4c30dd9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.404242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.404854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.405130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.408360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.408491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1154/2334] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:01.408586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.408763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.408763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.408763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.409070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.418703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1181/2335] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:01.419619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.428893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:01.430079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.430470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722ab16-eca9-4b92-a854-aa3097aaeca4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.435661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.436320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.436598 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.436867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.437521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:01.437817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.439425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.445226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1155/2336] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:01.446351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.449124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.453486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1182/2337] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:01.454737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.462595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537922f5-cd1d-4ca2-919b-d02ec901b1e2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.468379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.469324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.469436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff339776-a469-4e64-8cb1-4bdc9002f66f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.469930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.470402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] PATH_INFO: `/v3/projects/7f024797ea934eb3a25dc383bd4478ce/users/1f27dbae6e684f1c836f238de4b5ffdb/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.475013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.475623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.475901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.476525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.476911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:01.477216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.478931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.496042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1183/2338] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:01.497428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.505397 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f024797ea934eb3a25dc383bd4478ce, user_id=1f27dbae6e684f1c836f238de4b5ffdb, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.508139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.517744 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6098e91-595c-4f8c-883f-50c5ee32b4a5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.525573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.526180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.526456 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.526726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.527349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.529512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.530477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1156/2339] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:01 2022] PUT /identity/v3/projects/7f024797ea934eb3a25dc383bd4478ce/users/1f27dbae6e684f1c836f238de4b5ffdb/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:01.537384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.580500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1184/2340] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:01.580500 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.649070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436918e-6454-4d7c-88f2-ff996b579309 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.676771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b7066-b8d3-4c78-8d66-b969da746fae admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.681783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.682396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.682673 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.682944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.683582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.685212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.696169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1185/2341] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:01.696984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.704472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:01.709926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b14c8a-dbb0-4c83-9f33-076a51b9c85b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.722323 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.724479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.724775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.725216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.726414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.728407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.731760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.732469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.732758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.733030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.733660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.735548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.742537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:01.756731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1186/2342] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/users => generated 283 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:01.760098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.769428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b14ae-d055-468b-a3a5-ef92c99016e2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.769721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1157/2343] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/projects => generated 365 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:01.770705 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.843645 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.844752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.845188 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.845613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] PATH_INFO: `/v3/projects/7f024797ea934eb3a25dc383bd4478ce/users/1f27dbae6e684f1c836f238de4b5ffdb/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.848823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f024797ea934eb3a25dc383bd4478ce, user_id=1f27dbae6e684f1c836f238de4b5ffdb, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.851641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.851858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd470628-543b-4193-b717-6f488a6a2598 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.858213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e22262-7eb4-4d18-853f-185007ced1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.858824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e22262-7eb4-4d18-853f-185007ced1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.859102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e22262-7eb4-4d18-853f-185007ced1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.859373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e22262-7eb4-4d18-853f-185007ced1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.860049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e22262-7eb4-4d18-853f-185007ced1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.861644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e22262-7eb4-4d18-853f-185007ced1c1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:01.865267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1187/2344] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:01 2022] PUT /identity/v3/projects/7f024797ea934eb3a25dc383bd4478ce/users/1f27dbae6e684f1c836f238de4b5ffdb/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:01.876769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1158/2345] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:01.876956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:01.877081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.877081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.877081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.917762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] MFA Rules not processed for user `1f27dbae6e684f1c836f238de4b5ffdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:01.919929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:01.972209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913faf1f-c5ee-407c-b06f-a982dba64661 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:01.977335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:01.977947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:01.978227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:01.978503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:01.979287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:01.981236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.021612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1188/2346] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:02.022623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.038437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1159/2347] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:02.039448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.039853 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892a194-501d-4f14-9433-91f3e52cda4f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.047143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.047970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.048469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.049398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.050994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.052654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.054361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef868-32ac-41a5-b4cc-f6a786bbc2e1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.064721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1189/2348] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:02.066574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.066962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.067243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.067520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.067793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.068658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.069967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:02.070356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.082285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1160/2349] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:02.083243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.094401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef7e41-3ee9-4046-8a23-5c1c166ca8f0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.098757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.099425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.099702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.100552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] PATH_INFO: `/v3/projects/9c4a2e2df8844081a447f008d28ea006/users/17d6b047899a46d7b6ce1292bf5792ea/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.133288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4a2e2df8844081a447f008d28ea006, user_id=17d6b047899a46d7b6ce1292bf5792ea, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.136197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.168343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da47ff-81a0-4c15-ae36-0e425cd4c50d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.168539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1161/2350] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:02 2022] PUT /identity/v3/projects/9c4a2e2df8844081a447f008d28ea006/users/17d6b047899a46d7b6ce1292bf5792ea/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:02.168539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.196140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:02.242986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:02.263513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.264877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.265373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.265727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.291011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a243690-8300-409b-8fff-2daf28204603 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.300901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47994d1b-b448-437e-997e-bb6092c0f6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.302164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47994d1b-b448-437e-997e-bb6092c0f6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.302446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47994d1b-b448-437e-997e-bb6092c0f6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.302768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47994d1b-b448-437e-997e-bb6092c0f6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.303507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47994d1b-b448-437e-997e-bb6092c0f6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.305193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47994d1b-b448-437e-997e-bb6092c0f6ba admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.317615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1162/2351] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:02.322707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.323010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.323413 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.344826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.346845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.355496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] MFA Rules not processed for user `17d6b047899a46d7b6ce1292bf5792ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:02.356733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:02.357771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1190/2352] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:02.358400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.370984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5549b6-e8e4-4aec-9ee1-55c47ec1a377 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.382871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aeb5f-98cd-4b02-b7ce-a500f5d73eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.383523 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aeb5f-98cd-4b02-b7ce-a500f5d73eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.384060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aeb5f-98cd-4b02-b7ce-a500f5d73eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.384390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aeb5f-98cd-4b02-b7ce-a500f5d73eaf admin admin] PATH_INFO: `/v3/projects/141f2ca8eee3453bad5422d3f6169e5e/users/f853046d4e574976a74eaf0838097f16/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.404314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aeb5f-98cd-4b02-b7ce-a500f5d73eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=141f2ca8eee3453bad5422d3f6169e5e, user_id=f853046d4e574976a74eaf0838097f16, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.406994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aeb5f-98cd-4b02-b7ce-a500f5d73eaf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.427801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1191/2353] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:02 2022] PUT /identity/v3/projects/141f2ca8eee3453bad5422d3f6169e5e/users/f853046d4e574976a74eaf0838097f16/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:02.441600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.441935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.442218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.488691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1163/2354] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:02.495935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] MFA Rules not processed for user `f853046d4e574976a74eaf0838097f16`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:02.497238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:02.500849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.502384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:02.521082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae980e-1686-40d5-9462-5b2c47f17c72 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.592166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.592829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.593108 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.593518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1192/2355] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:02.593630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.610083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:02.610906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:02.650387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:02.669631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.671072 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.682607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1164/2356] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:02.683413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873d80-844e-426e-917b-22d7ff8e0553 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:02.687601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186b07-a2a3-4ba7-b991-7aeeefb83cac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:02.688296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186b07-a2a3-4ba7-b991-7aeeefb83cac service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:02.688576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186b07-a2a3-4ba7-b991-7aeeefb83cac service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:02.688848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186b07-a2a3-4ba7-b991-7aeeefb83cac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:02.698352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186b07-a2a3-4ba7-b991-7aeeefb83cac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:02.699770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186b07-a2a3-4ba7-b991-7aeeefb83cac service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:02.712077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1193/2357] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:03.229495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:03.241064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b09d6-c982-41d2-8d62-0e287b542682 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:03.247966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:03.248875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:03.249218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:03.249591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:03.252167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:03.266191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:03.342512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:03.344117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:03.355913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1165/2358] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:06.838633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3114eef-6234-4b18-8007-12dd5ed88f23 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:06.839074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3114eef-6234-4b18-8007-12dd5ed88f23 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:06.839163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3114eef-6234-4b18-8007-12dd5ed88f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:06.861742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d3114eef-6234-4b18-8007-12dd5ed88f23 None None] MFA Rules not processed for user `e4b36e6959444273872c300742d6ddb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:06.863138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d3114eef-6234-4b18-8007-12dd5ed88f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:06.969620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1194/2359] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:07.792968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:07.814128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0a7d9-d16d-4ae7-98e8-877c6463b6db service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:07.821316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:07.821927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:07.822223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:07.822473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:07.824243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:07.942444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:07.943900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:07.962943 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1166/2360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:08.067543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3565bae-1a1a-4224-a3a5-6818bc58c11e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.077979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3565bae-1a1a-4224-a3a5-6818bc58c11e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.078343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3565bae-1a1a-4224-a3a5-6818bc58c11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.138579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c3565bae-1a1a-4224-a3a5-6818bc58c11e None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:08.139740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c3565bae-1a1a-4224-a3a5-6818bc58c11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.249219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1195/2361] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:08.325127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:08.337802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:08.437253 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b141ddb-452c-4ffa-bee3-0112f7fed999 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:08.444855 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:08.445645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:08.446105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:08.447039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:08.459053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:08.464276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:08.483477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1167/2362] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:41:09.488594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250530e-ffec-405f-b0a2-3fafd7476a37 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:09.489218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250530e-ffec-405f-b0a2-3fafd7476a37 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:09.489218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250530e-ffec-405f-b0a2-3fafd7476a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:09.509627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3250530e-ffec-405f-b0a2-3fafd7476a37 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:09.510829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3250530e-ffec-405f-b0a2-3fafd7476a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:09.518671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1196/2363] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:09.531556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:09.543404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c97f01-cfe5-4142-9495-95aba8a0b744 service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:09.548506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:09.549107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:09.549586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:09.549934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:09.558642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:09.562355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:09.572981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1168/2364] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:41:09.917445 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cddbf-09ae-40a2-8aa7-a1882722e4b5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:09.917727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cddbf-09ae-40a2-8aa7-a1882722e4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:09.918185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cddbf-09ae-40a2-8aa7-a1882722e4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:09.943307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3a0cddbf-09ae-40a2-8aa7-a1882722e4b5 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:09.945087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a0cddbf-09ae-40a2-8aa7-a1882722e4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:09.965285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1197/2365] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:09.980244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:09.994826 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e6d6-97b5-49a9-8fb6-5d29edf32f54 service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:10.013329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:10.013956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.014236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.014507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.025731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:10.027322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:10.039526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1169/2366] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:41:10.526238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.527597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.528355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.570568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:10.573355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:10.613960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1198/2367] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:10.615292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:10.647696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641a947-36f7-4320-85ef-87e72eb62621 service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:10.654590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3215d-2c64-4ec2-88d3-1b6298b3ef9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:10.656500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3215d-2c64-4ec2-88d3-1b6298b3ef9b service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:10.656973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3215d-2c64-4ec2-88d3-1b6298b3ef9b service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:10.657926 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3215d-2c64-4ec2-88d3-1b6298b3ef9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:10.670757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3215d-2c64-4ec2-88d3-1b6298b3ef9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:10.672612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3215d-2c64-4ec2-88d3-1b6298b3ef9b service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:10.681486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:10.687029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1170/2368] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:41:11.197341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:11.268771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb0373-5bdb-465f-9a8d-df0899382476 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:11.273587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:11.274436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:11.274719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:11.274990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] PATH_INFO: `/v3/users/0cc25c9697d64e28970709fc6c0f35a4` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:11.276170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc25c9697d64e28970709fc6c0f35a4)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:11.278389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:11.283477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:11.337328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc25c9697d64e28970709fc6c0f35a4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:11.344183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc25c9697d64e28970709fc6c0f35a4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:11.344872 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc25c9697d64e28970709fc6c0f35a4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:11.354829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc25c9697d64e28970709fc6c0f35a4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:11.367920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc25c9697d64e28970709fc6c0f35a4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:11.374246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1199/2369] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:11 2022] DELETE /identity/v3/users/0cc25c9697d64e28970709fc6c0f35a4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:11.379919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:11.380645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:11.381319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:11.405280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:11.406565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:11.472732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1171/2370] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:11.484312 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:11.486436 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:11.564215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89da68f-c1dd-4b8f-8869-570a3062df6c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:11.569811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:11.570516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:11.570791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:11.571076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:11.592879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:11.594487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:11.608644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1200/2371] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:11.969905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:11.972342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:11.988799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef83c6-7a4d-411b-8868-d4b54c1fe7e7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:11.995238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:11.996052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:11.996387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:11.996745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] PATH_INFO: `/v3/projects/f213d7be9c1c4a0d8d2e4e8b4950deb6` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:12.003417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f213d7be9c1c4a0d8d2e4e8b4950deb6)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:12.003417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:12.065227 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f213d7be9c1c4a0d8d2e4e8b4950deb6', 'request_id': 'req-9b90a29f-d010-42c5-85a3-ff11be1fb493'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.076857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f213d7be9c1c4a0d8d2e4e8b4950deb6', 'request_id': 'req-9b90a29f-d010-42c5-85a3-ff11be1fb493'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.082866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f213d7be9c1c4a0d8d2e4e8b4950deb6', 'request_id': 'req-9b90a29f-d010-42c5-85a3-ff11be1fb493'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.098437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] The token cache is being invalidate because project f213d7be9c1c4a0d8d2e4e8b4950deb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:12.098969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.099730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.101722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1172/2372] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:11 2022] DELETE /identity/v3/projects/f213d7be9c1c4a0d8d2e4e8b4950deb6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:12.106614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:12.121345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7ff9b-d9a2-46bb-a157-ff89ea72ecd8 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:12.195327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:12.195994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:12.196284 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:12.196563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] PATH_INFO: `/v3/users/7df85dad4f054ed2ac6e7708e2c63de7` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:12.197711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df85dad4f054ed2ac6e7708e2c63de7)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:12.199699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:12.229855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:12.250999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df85dad4f054ed2ac6e7708e2c63de7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.262144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df85dad4f054ed2ac6e7708e2c63de7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.263910 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df85dad4f054ed2ac6e7708e2c63de7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.271274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df85dad4f054ed2ac6e7708e2c63de7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.283346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df85dad4f054ed2ac6e7708e2c63de7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.289698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1201/2373] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:12 2022] DELETE /identity/v3/users/7df85dad4f054ed2ac6e7708e2c63de7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:12.555529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:12.557118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:12.570733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90a29f-d010-42c5-85a3-ff11be1fb493 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:12.629488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:12.630128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:12.630404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:12.630677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] PATH_INFO: `/v3/projects/0cbe66ffbacb40f0b9dfada5461cec9b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:12.631792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cbe66ffbacb40f0b9dfada5461cec9b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:12.633722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:12.664223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:12.686299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cbe66ffbacb40f0b9dfada5461cec9b', 'request_id': 'req-16629032-bd20-4d18-9402-1ffaf34f9283'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.694935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cbe66ffbacb40f0b9dfada5461cec9b', 'request_id': 'req-16629032-bd20-4d18-9402-1ffaf34f9283'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.701086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cbe66ffbacb40f0b9dfada5461cec9b', 'request_id': 'req-16629032-bd20-4d18-9402-1ffaf34f9283'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.717567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] The token cache is being invalidate because project 0cbe66ffbacb40f0b9dfada5461cec9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:12.717857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.718494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.720365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1173/2374] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:12 2022] DELETE /identity/v3/projects/0cbe66ffbacb40f0b9dfada5461cec9b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:12.725458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:12.726776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:12.742314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2e3c-3b95-444d-a416-066066eac415 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:12.802534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:12.803072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:12.803352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:12.803655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] PATH_INFO: `/v3/users/1f27dbae6e684f1c836f238de4b5ffdb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:12.804830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f27dbae6e684f1c836f238de4b5ffdb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:12.806734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:12.858405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f27dbae6e684f1c836f238de4b5ffdb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.866259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f27dbae6e684f1c836f238de4b5ffdb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.866984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f27dbae6e684f1c836f238de4b5ffdb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.875257 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f27dbae6e684f1c836f238de4b5ffdb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.887882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-148ab59b-1c3d-4771-a81f-ad061f6e0585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f27dbae6e684f1c836f238de4b5ffdb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:12.894389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1202/2375] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:12 2022] DELETE /identity/v3/users/1f27dbae6e684f1c836f238de4b5ffdb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:13.195309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.196822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:13.211004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16629032-bd20-4d18-9402-1ffaf34f9283 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.278732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.279405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.279640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.279921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] PATH_INFO: `/v3/projects/7f024797ea934eb3a25dc383bd4478ce` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.281093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f024797ea934eb3a25dc383bd4478ce)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.283056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.333732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f024797ea934eb3a25dc383bd4478ce', 'request_id': 'req-93199719-16de-47a0-a04a-487464bc508b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:13.340120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f024797ea934eb3a25dc383bd4478ce', 'request_id': 'req-93199719-16de-47a0-a04a-487464bc508b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:13.345861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f024797ea934eb3a25dc383bd4478ce', 'request_id': 'req-93199719-16de-47a0-a04a-487464bc508b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:13.360060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] The token cache is being invalidate because project 7f024797ea934eb3a25dc383bd4478ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:13.360457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:13.361086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-93199719-16de-47a0-a04a-487464bc508b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:13.362798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1174/2376] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:13 2022] DELETE /identity/v3/projects/7f024797ea934eb3a25dc383bd4478ce => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:13.368532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.368812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.369081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.388785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:13.389992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:13.417736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:13.465658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1203/2377] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:13.476585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.477954 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.478796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.500250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:13.501455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:13.510475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1175/2378] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:13.514843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.525435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a918f-a8fc-4be7-afa4-b555a584fc68 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.530219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.530818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.531119 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.531393 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.532072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:13.532378 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.534597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.548126 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1204/2379] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:13.553063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.562456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38952c-c516-420e-b628-2bf72a6edf7a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.567709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.568345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.568624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.568893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.569525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.571437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.599293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1176/2380] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:13 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:13.603537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.615980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346af6d-87bd-400f-91a3-438921de0c41 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.674668 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.675301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.675565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.675818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.676606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.678488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.703350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1205/2381] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:13 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:13.704578 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.715362 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a131a8-ffff-4b00-b9d1-4dfba2be9b8a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.720051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.720672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.721564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.721842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.722485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.724120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.734074 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1177/2382] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:13.739067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.749657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ec0f-2445-478b-90b8-a572e6f224f5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.755531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5792d-1b56-4bbe-ac8b-3bcf6f5e5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.756176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5792d-1b56-4bbe-ac8b-3bcf6f5e5bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.756459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5792d-1b56-4bbe-ac8b-3bcf6f5e5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.756732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5792d-1b56-4bbe-ac8b-3bcf6f5e5bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.757526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5792d-1b56-4bbe-ac8b-3bcf6f5e5bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.759224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5792d-1b56-4bbe-ac8b-3bcf6f5e5bd7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.771606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1206/2383] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:13.775762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:13.782710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:13.787263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b9d31-61ec-4e62-83e0-cac97ad06ef0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:13.793710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:13.795257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.795540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.795830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] PATH_INFO: `/v3/projects/9543d8eeb7844b6482411e26be08b01e/users/40716140fd184782bf82197e64ced897/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.811768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9543d8eeb7844b6482411e26be08b01e, user_id=40716140fd184782bf82197e64ced897, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:13.814453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:13.826583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1178/2384] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:13 2022] PUT /identity/v3/projects/9543d8eeb7844b6482411e26be08b01e/users/40716140fd184782bf82197e64ced897/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:13.839320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:13.839603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:13.839877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:13.869123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] MFA Rules not processed for user `40716140fd184782bf82197e64ced897`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:13.870428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:13.944913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1207/2385] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:13.949589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.019773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651513-d145-44ae-b81a-d54c89a4b30e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.024706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.025322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.025602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.025876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.026521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.028784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.059373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1179/2386] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:13 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:14.063676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.075107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85551942-2472-4182-8d76-5c8185a87c38 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.092062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:14.134428 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.135041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.135326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.135600 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.136264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.138177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.164618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1208/2387] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:14 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:14.168564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.182719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5c35c-7c23-4fa2-b6a8-415f7cbea8c9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.189146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.190676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.190676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.191090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.191760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.193449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.209464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1180/2388] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:14.210135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.222127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d198c-8821-46f7-b97d-1c03a10e9a7e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.228238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.228912 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.229196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.229553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] PATH_INFO: `/v3/projects/a5fc58edc34f4b0b820d091bccc267af/users/64074ca296fd46049b938766cccfaab2/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.258380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fc58edc34f4b0b820d091bccc267af, user_id=64074ca296fd46049b938766cccfaab2, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.272365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.294474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1209/2389] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:14 2022] PUT /identity/v3/projects/a5fc58edc34f4b0b820d091bccc267af/users/64074ca296fd46049b938766cccfaab2/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:14.298172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.319794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfecbf3-eb32-493f-8146-c33f963cf418 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.383186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18336f54-50a4-4e5f-b070-924309427077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.383714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18336f54-50a4-4e5f-b070-924309427077 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.384012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18336f54-50a4-4e5f-b070-924309427077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.384289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18336f54-50a4-4e5f-b070-924309427077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.384931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18336f54-50a4-4e5f-b070-924309427077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.386525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18336f54-50a4-4e5f-b070-924309427077 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.397033 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1181/2390] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:14.401109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.412166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37899de5-67d2-468d-846a-85c9b3e4443c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.416799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.417444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.417722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.417988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] PATH_INFO: `/v3/projects/a5fc58edc34f4b0b820d091bccc267af/users/64074ca296fd46049b938766cccfaab2/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.419874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fc58edc34f4b0b820d091bccc267af, user_id=64074ca296fd46049b938766cccfaab2, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.422508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.433612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1210/2391] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:14 2022] PUT /identity/v3/projects/a5fc58edc34f4b0b820d091bccc267af/users/64074ca296fd46049b938766cccfaab2/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:14.444109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.444402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.444682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.477459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] MFA Rules not processed for user `64074ca296fd46049b938766cccfaab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:14.478614 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:14.526444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:14.573304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1182/2392] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:14.585241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.587194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:14.649747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0cba6-b626-4eb8-a9f6-99693041293c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.654026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976b609-23e7-4438-bf9d-a3da1911e7b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.654647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976b609-23e7-4438-bf9d-a3da1911e7b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.654926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976b609-23e7-4438-bf9d-a3da1911e7b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.655203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976b609-23e7-4438-bf9d-a3da1911e7b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.665113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976b609-23e7-4438-bf9d-a3da1911e7b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.666547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976b609-23e7-4438-bf9d-a3da1911e7b1 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.676699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1211/2393] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:14 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:14.823742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:14.890728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef769de5-904a-47fb-9e95-47b256282605 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:14.895375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:14.896152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.896434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.896708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] PATH_INFO: `/v3/users/40716140fd184782bf82197e64ced897` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:14.897829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40716140fd184782bf82197e64ced897)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:14.899739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:14.947114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40716140fd184782bf82197e64ced897'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:14.954882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40716140fd184782bf82197e64ced897'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:14.967758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40716140fd184782bf82197e64ced897'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:14.968505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40716140fd184782bf82197e64ced897'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:14.975896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40716140fd184782bf82197e64ced897'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:14.981741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1183/2394] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:14 2022] DELETE /identity/v3/users/40716140fd184782bf82197e64ced897 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:14.986155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:14.986664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:14.986816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.006890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:15.008134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.072698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1212/2395] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:15.082249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:15.083757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.163657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49aa4-cc9e-471a-95a1-67bb7862d086 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:15.167481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ed280-092a-4e60-b380-68b6afc2a69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:15.168127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ed280-092a-4e60-b380-68b6afc2a69e service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.168407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ed280-092a-4e60-b380-68b6afc2a69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.168682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ed280-092a-4e60-b380-68b6afc2a69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.179371 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ed280-092a-4e60-b380-68b6afc2a69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:15.180858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ed280-092a-4e60-b380-68b6afc2a69e service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:15.185102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:15.195344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1184/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:15.236027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:15.238321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.254360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7a74-5e0e-4bcf-818a-b3e11e28e5a3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:15.259063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:15.260710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.260710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.260710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] PATH_INFO: `/v3/users/ee66b7c264dd44db887898923c925dc8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.261846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee66b7c264dd44db887898923c925dc8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:15.264857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:15.279603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:15.329228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee66b7c264dd44db887898923c925dc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.340962 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee66b7c264dd44db887898923c925dc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.341882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee66b7c264dd44db887898923c925dc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.353845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee66b7c264dd44db887898923c925dc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.373705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee66b7c264dd44db887898923c925dc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.391939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.392355 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.392568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.395845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1213/2397] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:15 2022] DELETE /identity/v3/users/ee66b7c264dd44db887898923c925dc8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:15.414398 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:15.415565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.489854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1185/2398] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:15.501280 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:15.501590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.582346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db639-0f90-416e-b72b-f1df2c38038c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:15.586168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:15.586861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.587228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.587573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.603695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:15.605433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:15.620274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1214/2399] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:15.624175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:15.625848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:15.642353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189d7db-a05e-4160-a3d1-51694de67372 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:15.647448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:15.648128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.648927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.648927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] PATH_INFO: `/v3/projects/9543d8eeb7844b6482411e26be08b01e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.651702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9543d8eeb7844b6482411e26be08b01e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:15.657159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:15.732412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9543d8eeb7844b6482411e26be08b01e', 'request_id': 'req-e1a0f394-9d44-4733-a287-4419060dca4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.742164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9543d8eeb7844b6482411e26be08b01e', 'request_id': 'req-e1a0f394-9d44-4733-a287-4419060dca4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.752790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9543d8eeb7844b6482411e26be08b01e', 'request_id': 'req-e1a0f394-9d44-4733-a287-4419060dca4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.771604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] The token cache is being invalidate because project 9543d8eeb7844b6482411e26be08b01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:15.771901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.773332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.775152 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1186/2400] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:15 2022] DELETE /identity/v3/projects/9543d8eeb7844b6482411e26be08b01e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:15.779225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:15.792483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb1a56-c80f-4c22-a4ca-46e4f541ab76 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:15.866690 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:15.867444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:15.867796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:15.868234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] PATH_INFO: `/v3/users/64074ca296fd46049b938766cccfaab2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:15.869555 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64074ca296fd46049b938766cccfaab2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:15.871628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:15.913960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:15.934040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64074ca296fd46049b938766cccfaab2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.943768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64074ca296fd46049b938766cccfaab2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.944714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64074ca296fd46049b938766cccfaab2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.952374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64074ca296fd46049b938766cccfaab2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.968814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1cc938b3-194a-4eda-85d0-1e848ba48487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64074ca296fd46049b938766cccfaab2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:15.977024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:15.978550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1215/2401] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:15 2022] DELETE /identity/v3/users/64074ca296fd46049b938766cccfaab2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:15.985001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.015664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0f394-9d44-4733-a287-4419060dca4d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.126428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.127704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.128130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.128480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] PATH_INFO: `/v3/projects/fe5a43a5024141fc9c9977086b245550` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.129964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5a43a5024141fc9c9977086b245550)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.132155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.186426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:16.221740 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5a43a5024141fc9c9977086b245550', 'request_id': 'req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.231436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5a43a5024141fc9c9977086b245550', 'request_id': 'req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.242621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5a43a5024141fc9c9977086b245550', 'request_id': 'req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.270683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] The token cache is being invalidate because project fe5a43a5024141fc9c9977086b245550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:16.271127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.272806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-47acfb32-eb53-4c6c-ae0e-48e0d52fd413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.280056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1187/2402] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:15 2022] DELETE /identity/v3/projects/fe5a43a5024141fc9c9977086b245550 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:16.284654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.284654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.284654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.312076 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:16.315294 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.453845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1216/2403] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:16.463940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.463940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.463940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.467677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:16.469174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.527254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81be2-4148-4f8d-be10-2aa8ee9d0a24 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.529096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:16.531967 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.544988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.545863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.546210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.546548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] PATH_INFO: `/v3/projects/a5fc58edc34f4b0b820d091bccc267af` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.548304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fc58edc34f4b0b820d091bccc267af)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.548426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1188/2404] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:16.551449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.554094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:16.572749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bef8e-a0d3-4d4b-9f25-64b594caac09 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.582345 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.583045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.583328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.583692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.584509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:16.584868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.586733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.614765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1189/2405] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:16.619808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:16.637848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eee13c-4871-4ce1-b722-9353be98de14 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.659803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fc58edc34f4b0b820d091bccc267af', 'request_id': 'req-c4d4b529-d2d4-45ca-9e34-98a5bf249282'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.692350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fc58edc34f4b0b820d091bccc267af', 'request_id': 'req-c4d4b529-d2d4-45ca-9e34-98a5bf249282'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.704011 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fc58edc34f4b0b820d091bccc267af', 'request_id': 'req-c4d4b529-d2d4-45ca-9e34-98a5bf249282'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.726742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] The token cache is being invalidate because project a5fc58edc34f4b0b820d091bccc267af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:16.727146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.728166 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c4d4b529-d2d4-45ca-9e34-98a5bf249282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:16.733328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1217/2406] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:16 2022] DELETE /identity/v3/projects/a5fc58edc34f4b0b820d091bccc267af => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:16.746658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.747306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.747674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.748007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.748649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.751440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.751740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b5b76-8c2b-4ffb-bc8a-f4c981dec81c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.752095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b5b76-8c2b-4ffb-bc8a-f4c981dec81c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.752938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b5b76-8c2b-4ffb-bc8a-f4c981dec81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.799998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b83b5b76-8c2b-4ffb-bc8a-f4c981dec81c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:16.801274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b83b5b76-8c2b-4ffb-bc8a-f4c981dec81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.805258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1190/2407] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:16.808742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:16.811176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.864333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:16.912253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1218/2408] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:16.915761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.916210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.916602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.925204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153627d-5c6c-4179-8774-c747aff7a655 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.930679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.931298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.931582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.931863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.932530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.934487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.937772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:16.940402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:16.950616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1219/2409] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:16.954761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:16.965136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85965cb1-4eea-4f34-8ea6-cae9b9522b59 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.968112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1191/2410] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:16.969085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:16.970066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.971064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.971064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.971583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.972487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:16.972791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.974564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.979521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf27a2-060f-4a27-a357-2f8391bac9f5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:16.984167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:16.984779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:16.985064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:16.985342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:16.985990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:16.988161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:16.989399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1220/2411] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:41:16.989864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:16.994127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.001698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1192/2412] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:17.006151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.007184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757ef22-efb9-447d-a4ba-36a8c5ca456d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.012566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.013175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.013461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.013726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.014446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.016307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.021416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f049d-7ca9-4048-a338-48a6adf18633 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.026965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.028165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.028702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.029176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.030109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.032241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.044426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1193/2413] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:17.048255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1221/2414] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:16 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:17.048661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.053030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.060266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d36c2a-a31f-49a7-90f0-dbc94eeba172 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.130647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7db0-7874-4946-a3df-cedf06d17406 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.135891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.136567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.136989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.137105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738397af-b74c-4165-8fa1-6314e1ea102b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.137381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.137859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738397af-b74c-4165-8fa1-6314e1ea102b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.138067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738397af-b74c-4165-8fa1-6314e1ea102b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.138183 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.138327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738397af-b74c-4165-8fa1-6314e1ea102b admin admin] PATH_INFO: `/v3/projects/297efa814ad849888e4587aa971bc067/users/ea40106c284c4492b2e6f8411d4372c8/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.140122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.159734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738397af-b74c-4165-8fa1-6314e1ea102b admin admin] RBAC: Authorizing `identity:create_grant(project_id=297efa814ad849888e4587aa971bc067, user_id=ea40106c284c4492b2e6f8411d4372c8, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.162415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738397af-b74c-4165-8fa1-6314e1ea102b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.163797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1222/2415] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:17 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:17.167727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.179738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1194/2416] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:17 2022] PUT /identity/v3/projects/297efa814ad849888e4587aa971bc067/users/ea40106c284c4492b2e6f8411d4372c8/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:17.180239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f6edf-3a8d-4b64-b583-24ddf776ea5f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.191906 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.192147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.192552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.222423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] MFA Rules not processed for user `ea40106c284c4492b2e6f8411d4372c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:17.223657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:17.241213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.241828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.242196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.242389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.243037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.244700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.255697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1223/2417] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:17.259425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.270065 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea1b9e-0b7c-43b3-b9cd-75d512d303e4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.274783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.275380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.275666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.275961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.276612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.278469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.289396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1224/2418] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:17.293685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.302564 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1195/2419] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:17.307983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f17db-8356-45f9-9f00-e96ebdbbf18b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.309987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.311361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:17.312762 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3c47e-723f-46e8-9e7b-7ee63acd4474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.313421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c47e-723f-46e8-9e7b-7ee63acd4474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.313699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c47e-723f-46e8-9e7b-7ee63acd4474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.313976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3c47e-723f-46e8-9e7b-7ee63acd4474 admin admin] PATH_INFO: `/v3/projects/b9e2651cf64b43beaacc1fa7859f64a6/users/8b661051e526423a920d0b490872663b/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.317624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:17.333791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3c47e-723f-46e8-9e7b-7ee63acd4474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e2651cf64b43beaacc1fa7859f64a6, user_id=8b661051e526423a920d0b490872663b, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.336528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3c47e-723f-46e8-9e7b-7ee63acd4474 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.352699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1225/2420] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:17 2022] PUT /identity/v3/projects/b9e2651cf64b43beaacc1fa7859f64a6/users/8b661051e526423a920d0b490872663b/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:17.364440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.364752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.365025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.376592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d61fa8-9123-49f5-8acb-b26ccc290a60 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.385828 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:17.395259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] MFA Rules not processed for user `8b661051e526423a920d0b490872663b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:17.396789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:17.433895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19208163-798b-4975-9839-ee957110b992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.434506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19208163-798b-4975-9839-ee957110b992 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.434786 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19208163-798b-4975-9839-ee957110b992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.435065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19208163-798b-4975-9839-ee957110b992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.469252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1226/2421] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:17.474497 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.503127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19208163-798b-4975-9839-ee957110b992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.504662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19208163-798b-4975-9839-ee957110b992 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.525673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1196/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:17.559859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68217b0f-c0bf-498a-ba92-aac2796efb4f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.565522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.566149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.566432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.566708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.567347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.569382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.613765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1227/2423] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:17 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:17.615039 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19208163-798b-4975-9839-ee957110b992 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.631639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19208163-798b-4975-9839-ee957110b992 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.708605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.709221 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.709506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.709781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.710422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.712508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.744170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1197/2424] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:17 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:17.750475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.763848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa17d5-01b6-47d1-b2d6-938c5f429d8a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.770124 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.770733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.771015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.771287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.771935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.773558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.786107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1228/2425] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:17.792183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.806352 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f835-f3fe-4689-bc9b-75963e3e54e7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:17.816176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:17.817173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:17.820135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:17.820135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] PATH_INFO: `/v3/projects/1f8396ef0f79467fbaebed083c99fe7c/users/5ae9e9751daa420786311b86b3cb91e4/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:17.838336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8396ef0f79467fbaebed083c99fe7c, user_id=5ae9e9751daa420786311b86b3cb91e4, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:17.841916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:17.868811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1198/2426] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:17 2022] PUT /identity/v3/projects/1f8396ef0f79467fbaebed083c99fe7c/users/5ae9e9751daa420786311b86b3cb91e4/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:17.869170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:17.898473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6e8d8-bc3e-44fe-82d8-bca9c32e7e5f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:18.012674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:18.044324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c740-4bbc-4201-a1c5-8ccc382a8d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:18.047425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c740-4bbc-4201-a1c5-8ccc382a8d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:18.048210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c740-4bbc-4201-a1c5-8ccc382a8d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:18.049161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c740-4bbc-4201-a1c5-8ccc382a8d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:18.050667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c740-4bbc-4201-a1c5-8ccc382a8d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:18.056106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c740-4bbc-4201-a1c5-8ccc382a8d4c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:18.093390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1229/2427] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:18.094042 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:18.110919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eec0c-92e2-476b-b243-8d6d66c2ecee admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:18.120168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:18.120919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:18.121242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:18.121387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] PATH_INFO: `/v3/projects/1f8396ef0f79467fbaebed083c99fe7c/users/5ae9e9751daa420786311b86b3cb91e4/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:18.129010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8396ef0f79467fbaebed083c99fe7c, user_id=5ae9e9751daa420786311b86b3cb91e4, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:18.132664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:18.167976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1199/2428] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:18 2022] PUT /identity/v3/projects/1f8396ef0f79467fbaebed083c99fe7c/users/5ae9e9751daa420786311b86b3cb91e4/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:18.177783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:18.179012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:18.179494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:18.255840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] MFA Rules not processed for user `5ae9e9751daa420786311b86b3cb91e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:18.257190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:18.392006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1230/2429] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:18.403890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:18.452215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:18.538041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169f709-3267-461a-9d32-37edb8461158 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:18.541951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c57324-77a6-42a9-82ee-f7a703b2193f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:18.542675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c57324-77a6-42a9-82ee-f7a703b2193f service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:18.542974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c57324-77a6-42a9-82ee-f7a703b2193f service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:18.543332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c57324-77a6-42a9-82ee-f7a703b2193f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:18.558764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c57324-77a6-42a9-82ee-f7a703b2193f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:18.560832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c57324-77a6-42a9-82ee-f7a703b2193f service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:18.572781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1200/2430] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:18.993992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:19.060826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ac537-f7c1-4c70-a2bf-66932f17539d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:19.065726 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:19.066357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:19.066631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:19.066908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] PATH_INFO: `/v3/users/8b661051e526423a920d0b490872663b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:19.068050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b661051e526423a920d0b490872663b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:19.069948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:19.130212 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b661051e526423a920d0b490872663b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.137200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b661051e526423a920d0b490872663b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.137906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b661051e526423a920d0b490872663b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.145993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b661051e526423a920d0b490872663b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.151824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:19.162856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b661051e526423a920d0b490872663b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.172205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1231/2431] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:18 2022] DELETE /identity/v3/users/8b661051e526423a920d0b490872663b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:19.176206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:19.176493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:19.176895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:19.201211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:19.202509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:19.270280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1201/2432] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:19.281838 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:19.284058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:19.357847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c0734-1461-44d2-9da6-0b81884a670f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:19.370041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:19.371153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:19.371628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:19.372335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:19.385405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:19.387565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:19.404107 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1232/2433] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:41:19.901847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:19.903830 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:19.920183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d8df-f7b4-499b-977d-d6a553a3d6e9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:19.929419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:19.930099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:19.930378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:19.930682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] PATH_INFO: `/v3/projects/b9e2651cf64b43beaacc1fa7859f64a6` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:19.931756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e2651cf64b43beaacc1fa7859f64a6)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:19.933714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:19.984600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e2651cf64b43beaacc1fa7859f64a6', 'request_id': 'req-478a6872-de69-4a87-a515-2b3153968ef7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.991312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e2651cf64b43beaacc1fa7859f64a6', 'request_id': 'req-478a6872-de69-4a87-a515-2b3153968ef7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:19.996866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e2651cf64b43beaacc1fa7859f64a6', 'request_id': 'req-478a6872-de69-4a87-a515-2b3153968ef7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.009882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] The token cache is being invalidate because project b9e2651cf64b43beaacc1fa7859f64a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:20.010169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.010939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.012504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1202/2434] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:19 2022] DELETE /identity/v3/projects/b9e2651cf64b43beaacc1fa7859f64a6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:20.016507 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:20.028977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5ba1f-89a8-4237-9c85-dca46a357f6d service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:20.091429 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:20.092202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.092516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.092812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] PATH_INFO: `/v3/users/5ae9e9751daa420786311b86b3cb91e4` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.094062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae9e9751daa420786311b86b3cb91e4)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:20.096809 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:20.144688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae9e9751daa420786311b86b3cb91e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.151708 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae9e9751daa420786311b86b3cb91e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.152423 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae9e9751daa420786311b86b3cb91e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.160963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae9e9751daa420786311b86b3cb91e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.172777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-030799dc-741a-47f9-8af6-ffd5ebe6434b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae9e9751daa420786311b86b3cb91e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.178682 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1233/2435] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:20 2022] DELETE /identity/v3/users/5ae9e9751daa420786311b86b3cb91e4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:20.449511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:20.450964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:20.475456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6872-de69-4a87-a515-2b3153968ef7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:20.526127 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:20.541734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:20.542302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.542581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.542853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] PATH_INFO: `/v3/projects/1f8396ef0f79467fbaebed083c99fe7c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.544032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f8396ef0f79467fbaebed083c99fe7c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:20.545965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:20.593466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f8396ef0f79467fbaebed083c99fe7c', 'request_id': 'req-9fbfe916-2d3a-4389-a76a-f21f047b8251'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.599694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f8396ef0f79467fbaebed083c99fe7c', 'request_id': 'req-9fbfe916-2d3a-4389-a76a-f21f047b8251'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.605270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f8396ef0f79467fbaebed083c99fe7c', 'request_id': 'req-9fbfe916-2d3a-4389-a76a-f21f047b8251'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.618375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] The token cache is being invalidate because project 1f8396ef0f79467fbaebed083c99fe7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:20.618666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.619273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9fbfe916-2d3a-4389-a76a-f21f047b8251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:20.621058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1203/2436] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:20 2022] DELETE /identity/v3/projects/1f8396ef0f79467fbaebed083c99fe7c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:20.626580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.627785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.627785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.647056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:20.648270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:20.652395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:20.723888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1234/2437] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:20.725492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.725775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.726051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.749153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:20.750357 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:20.760049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1204/2438] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:20.772693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:20.775265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d42d1-403a-4844-86a0-17c37ccff52d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:20.780686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:20.781299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.781610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.781886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.782607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:20.782876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:20.785290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:20.797799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1235/2439] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:20.803259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:20.812571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef53ea9-e4f6-4676-bedf-7671b4fcb867 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:20.819169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:20.819779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.820117 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.820389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.821157 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:20.822892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:20.857252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1205/2440] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:20 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:20.861179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:20.872211 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d12b27-9ad7-444c-aad8-170c9cecba7d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:20.934238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:20.934847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.935127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.935402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.936060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:20.937978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:20.963330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1236/2441] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:20 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:20.964488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:20.975966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f12d5-c9b0-4661-ad96-75a6de7d9500 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:20.980547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:20.981153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:20.981432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:20.981708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:20.982358 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:20.984180 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:20.994939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1206/2442] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:20.999067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.009771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401367d1-ecb6-46ff-81f8-b56927043a2a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.014328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000490ae-7482-4c38-87c3-49d63109f427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.014918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000490ae-7482-4c38-87c3-49d63109f427 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.015195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000490ae-7482-4c38-87c3-49d63109f427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.015470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000490ae-7482-4c38-87c3-49d63109f427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.016154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000490ae-7482-4c38-87c3-49d63109f427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.018383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000490ae-7482-4c38-87c3-49d63109f427 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.033010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1237/2443] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:21.034314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.044428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5aed0-9725-49c8-a8d5-a3a778870492 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.049916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.050526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.050807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.051106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] PATH_INFO: `/v3/projects/18526e5ec7904365920c7ad99ab3f9da/users/2a0aa56ae34b4ff0a0305bb20743b877/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.066461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=18526e5ec7904365920c7ad99ab3f9da, user_id=2a0aa56ae34b4ff0a0305bb20743b877, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.069120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.082815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1207/2444] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:21 2022] PUT /identity/v3/projects/18526e5ec7904365920c7ad99ab3f9da/users/2a0aa56ae34b4ff0a0305bb20743b877/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:21.092958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.093246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.093522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.121865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] MFA Rules not processed for user `2a0aa56ae34b4ff0a0305bb20743b877`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:21.123081 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:21.188948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1238/2445] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:21.194248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.259523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986500b-f963-4489-bbe4-2fd016a4a12a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.264447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.265057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.265339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.265618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.266275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.268958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.290504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:21.299901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1208/2446] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:21 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:21.303381 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.309851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:21.314308 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70824b1-75e2-46fc-8bcd-d16937beed36 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.370991 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.371588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.371867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.372169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.372790 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.374697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.395324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1239/2447] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:21 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:21.399498 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.410613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0ea4-d6ab-4b45-aea3-d6eb70c046a8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.415287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.416008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.416314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.416710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.417548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.420679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.437504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1209/2448] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:21.441330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.451780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e599bc-3dc0-4b96-8e75-5c9b789e23e6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.457772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.458439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.458723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.459399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] PATH_INFO: `/v3/projects/be1e1e27185346d58f7257bd38d548ca/users/7db40f9384a849e4ba6aac5f14145b9e/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.482922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1e1e27185346d58f7257bd38d548ca, user_id=7db40f9384a849e4ba6aac5f14145b9e, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.486102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.500453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1240/2449] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:21 2022] PUT /identity/v3/projects/be1e1e27185346d58f7257bd38d548ca/users/7db40f9384a849e4ba6aac5f14145b9e/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:41:21.505703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.520318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54001e6-3d39-4618-a2f0-ad6d334d1f98 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.592148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d9491-3cb4-43cf-b9ed-1fbfff34654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.592979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d9491-3cb4-43cf-b9ed-1fbfff34654e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.593404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d9491-3cb4-43cf-b9ed-1fbfff34654e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.593768 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d9491-3cb4-43cf-b9ed-1fbfff34654e admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.594503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d9491-3cb4-43cf-b9ed-1fbfff34654e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.596621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d9491-3cb4-43cf-b9ed-1fbfff34654e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.610860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1210/2450] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:21.613016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.624706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f2293-d6c4-466a-a6aa-bde0a46872d8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.632296 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.633330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.633946 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.634664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] PATH_INFO: `/v3/projects/be1e1e27185346d58f7257bd38d548ca/users/7db40f9384a849e4ba6aac5f14145b9e/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.637199 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1e1e27185346d58f7257bd38d548ca, user_id=7db40f9384a849e4ba6aac5f14145b9e, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.640528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.653594 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1241/2451] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:21 2022] PUT /identity/v3/projects/be1e1e27185346d58f7257bd38d548ca/users/7db40f9384a849e4ba6aac5f14145b9e/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:21.664209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.664499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.664776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.696715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] MFA Rules not processed for user `7db40f9384a849e4ba6aac5f14145b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:21.698014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:21.768317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1211/2452] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:21.780024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.781889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:21.840623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063f25e-6b00-4b38-b3d9-9de6d740ec57 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.844352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.844956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.845238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.846016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:21.855742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:21.857190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:21.868162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1242/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:21.945804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:21.956223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c506a2-afd8-4658-bef0-55663634a1fd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:21.959919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e016a-d191-47c2-b99c-910e245c4ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:21.960577 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e016a-d191-47c2-b99c-910e245c4ecd service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:21.960856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e016a-d191-47c2-b99c-910e245c4ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:21.961154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e016a-d191-47c2-b99c-910e245c4ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:22.027901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e016a-d191-47c2-b99c-910e245c4ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:22.030049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e016a-d191-47c2-b99c-910e245c4ecd service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:22.041145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1212/2454] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:22.314072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:22.387686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea518e-d7dc-4bfd-83a3-d16a3ff279bf service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:22.392801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:22.394163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:22.394516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:22.394882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] PATH_INFO: `/v3/users/2a0aa56ae34b4ff0a0305bb20743b877` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:22.396110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a0aa56ae34b4ff0a0305bb20743b877)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:22.398482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:22.400425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:22.467583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a0aa56ae34b4ff0a0305bb20743b877'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:22.484522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a0aa56ae34b4ff0a0305bb20743b877'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:22.485714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a0aa56ae34b4ff0a0305bb20743b877'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:22.498409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a0aa56ae34b4ff0a0305bb20743b877'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:22.515477 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a0aa56ae34b4ff0a0305bb20743b877'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:22.523755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1243/2455] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:22 2022] DELETE /identity/v3/users/2a0aa56ae34b4ff0a0305bb20743b877 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:22.541862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:22.542521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:22.543080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:22.570998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:22.578545 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:22.580632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:22.654106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1213/2456] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:22.690446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:22.692874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:22.758623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b182b-38cc-4eae-9e87-b3146331cf95 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:22.762210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:22.762817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:22.763093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:22.763368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:22.774766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:22.776219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:22.786813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1244/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:23.079798 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:23.081473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:23.093337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82073f15-e69f-41ef-8017-28ef9ece5627 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:23.097994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:23.098627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:23.099070 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:23.099347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] PATH_INFO: `/v3/projects/18526e5ec7904365920c7ad99ab3f9da` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:23.100531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18526e5ec7904365920c7ad99ab3f9da)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:23.102488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:23.163075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18526e5ec7904365920c7ad99ab3f9da', 'request_id': 'req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.169974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18526e5ec7904365920c7ad99ab3f9da', 'request_id': 'req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.176378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18526e5ec7904365920c7ad99ab3f9da', 'request_id': 'req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.190514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] The token cache is being invalidate because project 18526e5ec7904365920c7ad99ab3f9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:23.190807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.191469 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.193022 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1214/2458] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:23 2022] DELETE /identity/v3/projects/18526e5ec7904365920c7ad99ab3f9da => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:23.197121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:23.208135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aad138-5574-4157-b169-0d45c9e9885c service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:23.280480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:23.281194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:23.281540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:23.281812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] PATH_INFO: `/v3/users/7db40f9384a849e4ba6aac5f14145b9e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:23.283470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db40f9384a849e4ba6aac5f14145b9e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:23.285566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:23.309572 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:23.335686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db40f9384a849e4ba6aac5f14145b9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.342654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db40f9384a849e4ba6aac5f14145b9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.343319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db40f9384a849e4ba6aac5f14145b9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.351469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db40f9384a849e4ba6aac5f14145b9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.364820 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64c7f453-f048-45eb-a5a7-532ff7110d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db40f9384a849e4ba6aac5f14145b9e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.370837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1245/2459] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:23 2022] DELETE /identity/v3/users/7db40f9384a849e4ba6aac5f14145b9e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:23.682418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:23.684576 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:23.740975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c66c3-0a0f-4144-ad5c-f00a0b7084c6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:23.805178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:23.805818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:23.806210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:23.806490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] PATH_INFO: `/v3/projects/be1e1e27185346d58f7257bd38d548ca` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:23.807622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1e1e27185346d58f7257bd38d548ca)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:23.809568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:23.843518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:23.858678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1e1e27185346d58f7257bd38d548ca', 'request_id': 'req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.866075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1e1e27185346d58f7257bd38d548ca', 'request_id': 'req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.871600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1e1e27185346d58f7257bd38d548ca', 'request_id': 'req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.885824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] The token cache is being invalidate because project be1e1e27185346d58f7257bd38d548ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:23.886119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.886763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-162aeea1-c95a-4cc6-b61a-0fe42a213d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:23.888499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1215/2460] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:23 2022] DELETE /identity/v3/projects/be1e1e27185346d58f7257bd38d548ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:23.893243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:23.893723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:23.893831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:23.913425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:23.914615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:23.977158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1246/2461] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:23.985753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:23.986036 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:23.986884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.006464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:24.007647 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:24.015346 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1216/2462] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:24.019596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.029372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f309267-4f42-4cf8-8f0c-c3f209add706 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.034395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.034992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.035268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.035537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.036208 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:24.036510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.038105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.050410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1247/2463] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:24.054735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.064175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369b91b-9116-42fa-890b-ae9a6569b91c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.069989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.070685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.070924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.071254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.072139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.074053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.102164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1217/2464] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:24 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:24.106316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.117041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79956-9807-4f91-adf2-3328719128e3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.172714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.173311 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.173621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.174471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.175147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.177072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.197935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1248/2465] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:24 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:24.202021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.213773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a912d-ebb2-48d3-8d97-a1867d29c4b5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.218455 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.219048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.219330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.219604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.220291 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.222259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.232384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1218/2466] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:24.236556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.247527 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8465c-94a8-4dce-a2b5-e2cec80b7adf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.253093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9e4fc-6528-4ea0-9f85-2086dd4f6dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.253773 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e4fc-6528-4ea0-9f85-2086dd4f6dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.254054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e4fc-6528-4ea0-9f85-2086dd4f6dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.254329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9e4fc-6528-4ea0-9f85-2086dd4f6dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.254978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9e4fc-6528-4ea0-9f85-2086dd4f6dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.256757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9e4fc-6528-4ea0-9f85-2086dd4f6dd0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.269193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1249/2467] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:24.273696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.284689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3e352-764e-4146-a08f-b192295f5a8c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.289750 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.290415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.290699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.290976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] PATH_INFO: `/v3/projects/7d48b729ba2d4e08ae78bc80bfce1bf5/users/0001a9bc1bb043b886d7c1bceea1a7bd/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.306298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d48b729ba2d4e08ae78bc80bfce1bf5, user_id=0001a9bc1bb043b886d7c1bceea1a7bd, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.309034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.323790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1219/2468] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:24 2022] PUT /identity/v3/projects/7d48b729ba2d4e08ae78bc80bfce1bf5/users/0001a9bc1bb043b886d7c1bceea1a7bd/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:24.335434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.335805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.336185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.349533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:24.365919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] MFA Rules not processed for user `0001a9bc1bb043b886d7c1bceea1a7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:24.367310 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:24.436791 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1250/2469] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:24.448611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.450479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:24.511666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782816-59ec-421b-815c-d886fd07ffa9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.515331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc15d02-8618-4c8b-9378-236d2a059c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.515957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc15d02-8618-4c8b-9378-236d2a059c61 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.516258 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc15d02-8618-4c8b-9378-236d2a059c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.516531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc15d02-8618-4c8b-9378-236d2a059c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.525746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc15d02-8618-4c8b-9378-236d2a059c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.527244 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc15d02-8618-4c8b-9378-236d2a059c61 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.537859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1220/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:24.559894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.626027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f3d5e-98c5-4ee4-afff-c1d4cda83906 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.631502 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.632172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.632452 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.632724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] PATH_INFO: `/v3/users/0001a9bc1bb043b886d7c1bceea1a7bd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.633850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0001a9bc1bb043b886d7c1bceea1a7bd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.635817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.681059 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0001a9bc1bb043b886d7c1bceea1a7bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:24.687725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0001a9bc1bb043b886d7c1bceea1a7bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:24.688506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0001a9bc1bb043b886d7c1bceea1a7bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:24.697389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0001a9bc1bb043b886d7c1bceea1a7bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:24.709528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0001a9bc1bb043b886d7c1bceea1a7bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:24.715357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1251/2471] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:24 2022] DELETE /identity/v3/users/0001a9bc1bb043b886d7c1bceea1a7bd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:24.719808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.720142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.720937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.743468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:24.744862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:24.818804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:24.829084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1221/2472] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:41:24.839239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:24.840819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:24.859372 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:24.907830 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ad5f-ff2e-4dee-92c5-e32801390d59 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:24.911780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76cf46-3fb8-4ce9-a2ba-f15add7ae4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:24.912417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76cf46-3fb8-4ce9-a2ba-f15add7ae4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:24.912694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76cf46-3fb8-4ce9-a2ba-f15add7ae4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:24.912967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76cf46-3fb8-4ce9-a2ba-f15add7ae4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:24.922476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76cf46-3fb8-4ce9-a2ba-f15add7ae4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:24.923918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76cf46-3fb8-4ce9-a2ba-f15add7ae4d2 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:24.933573 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1252/2473] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:25.203619 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.205385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:25.218061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e03ec5-bb18-4f39-931b-b41acd7e93a6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.223593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.224594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.224901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.225199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] PATH_INFO: `/v3/projects/7d48b729ba2d4e08ae78bc80bfce1bf5` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.227531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d48b729ba2d4e08ae78bc80bfce1bf5)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.230620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.285664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d48b729ba2d4e08ae78bc80bfce1bf5', 'request_id': 'req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:25.291747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d48b729ba2d4e08ae78bc80bfce1bf5', 'request_id': 'req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:25.297336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d48b729ba2d4e08ae78bc80bfce1bf5', 'request_id': 'req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:25.309888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] The token cache is being invalidate because project 7d48b729ba2d4e08ae78bc80bfce1bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:25.310172 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:25.310796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ab04a68-7d8d-41c2-bd66-f8e702fc5fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:25.312295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1222/2474] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:25 2022] DELETE /identity/v3/projects/7d48b729ba2d4e08ae78bc80bfce1bf5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:25.317964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.318247 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.318522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.337567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:25.338787 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:25.395929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1253/2475] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:25.403797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.404135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.404439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.424276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:25.425581 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:25.434106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1223/2476] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:25.438367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.448577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf3b5f-49fc-4398-9e23-828bde1c9a2c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.453532 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.453737 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.454013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.454285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.454979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:25.455281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.456918 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.468413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1254/2477] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:25.473417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.483089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859283-3ee8-4600-a057-7b5908e55966 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.489344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.489949 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.490222 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.490490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.491125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.493062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.528169 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1224/2478] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:25 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:25.529376 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.541579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf3ca5-91d0-489e-ab62-8e188a9e46b4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.624870 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.625491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.625772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.626046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.626828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.629084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.657453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1255/2479] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:25 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:25.659585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.672324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d330a-c62f-4806-a547-8c24efc0efab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.678356 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.678920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.679193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.679497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.680139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.681720 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.696985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1225/2480] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:25.699330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.710859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1d376-c080-47f8-8bfc-0e49c8767c85 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.715730 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fbb77-dcbc-43fc-a09d-947d7fe38588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.716371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fbb77-dcbc-43fc-a09d-947d7fe38588 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.716655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fbb77-dcbc-43fc-a09d-947d7fe38588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.716931 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fbb77-dcbc-43fc-a09d-947d7fe38588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.717606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fbb77-dcbc-43fc-a09d-947d7fe38588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.719220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fbb77-dcbc-43fc-a09d-947d7fe38588 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.734421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1256/2481] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:25.745743 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.756797 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49f9cf-9c78-4834-8a50-98aaf148375d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.761824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.762480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.762760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.763034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] PATH_INFO: `/v3/projects/f34ec21f18b14f59a716fe013a75df88/users/20a50779ddbb41e9bca9a7f82d635b1b/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.789880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34ec21f18b14f59a716fe013a75df88, user_id=20a50779ddbb41e9bca9a7f82d635b1b, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:25.792566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:25.806223 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1226/2482] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:25 2022] PUT /identity/v3/projects/f34ec21f18b14f59a716fe013a75df88/users/20a50779ddbb41e9bca9a7f82d635b1b/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:25.817709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.818100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.818450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:25.848365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] MFA Rules not processed for user `20a50779ddbb41e9bca9a7f82d635b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:25.849595 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:25.862524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:25.916379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1257/2483] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:25.927803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:25.929658 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:25.987761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee4ca5-c443-4342-9a2e-3af5cfeb83f1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:25.992038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b120-b076-427f-813b-fa736b482a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:25.992641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b120-b076-427f-813b-fa736b482a0b service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:25.992914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b120-b076-427f-813b-fa736b482a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:25.993516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b120-b076-427f-813b-fa736b482a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:26.006392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b120-b076-427f-813b-fa736b482a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:26.007836 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b120-b076-427f-813b-fa736b482a0b service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:26.018149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1227/2484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:35.913031 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=21 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:36.013686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:38.799223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:38.859508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92105f24-d5b3-49d6-bacc-e82b51d9646e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:38.864768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:38.865419 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:38.865693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:38.865961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] PATH_INFO: `/v3/users/20a50779ddbb41e9bca9a7f82d635b1b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:38.867118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a50779ddbb41e9bca9a7f82d635b1b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:38.869090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:38.916067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a50779ddbb41e9bca9a7f82d635b1b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:38.924606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a50779ddbb41e9bca9a7f82d635b1b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:38.925340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a50779ddbb41e9bca9a7f82d635b1b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:38.934170 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a50779ddbb41e9bca9a7f82d635b1b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:38.948664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a50779ddbb41e9bca9a7f82d635b1b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:38.955564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1258/2485] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:38 2022] DELETE /identity/v3/users/20a50779ddbb41e9bca9a7f82d635b1b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:38.960437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:38.960891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:38.960891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:38.982426 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:38.983632 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.061073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1228/2486] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:39.075974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:39.077446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.168948 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245b84d-82b6-42be-9f31-4b0707e928a2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:39.174235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade740b2-d1b1-48d9-ad29-f5cd86ea0798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:39.175465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade740b2-d1b1-48d9-ad29-f5cd86ea0798 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.175765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade740b2-d1b1-48d9-ad29-f5cd86ea0798 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.176115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade740b2-d1b1-48d9-ad29-f5cd86ea0798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.192616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade740b2-d1b1-48d9-ad29-f5cd86ea0798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:39.195389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade740b2-d1b1-48d9-ad29-f5cd86ea0798 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:39.207447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1259/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:39.217751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:39.219289 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.240376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21000d-5157-42e4-b2d7-26cdd7f14f6a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:39.247568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:39.248415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.248796 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.249909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] PATH_INFO: `/v3/users/79aa10274b604aa4af14fbe1b404cfce` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.251385 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79aa10274b604aa4af14fbe1b404cfce)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:39.253732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:39.321602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79aa10274b604aa4af14fbe1b404cfce'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.330068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79aa10274b604aa4af14fbe1b404cfce'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.343085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79aa10274b604aa4af14fbe1b404cfce'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.344148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79aa10274b604aa4af14fbe1b404cfce'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.352717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79aa10274b604aa4af14fbe1b404cfce'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.360865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1229/2488] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:39 2022] DELETE /identity/v3/users/79aa10274b604aa4af14fbe1b404cfce => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:39.365783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.366152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.366505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.397900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:39.399332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.420022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:39.486382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1260/2489] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:39.504147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:39.506417 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.586655 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9a945-3536-4572-bc4b-edef6d8ca5e4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:39.596476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a2dff-d84d-4f6a-b4de-c8eaa3500aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:39.597119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a2dff-d84d-4f6a-b4de-c8eaa3500aca service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.597390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a2dff-d84d-4f6a-b4de-c8eaa3500aca service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.597667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a2dff-d84d-4f6a-b4de-c8eaa3500aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.607748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a2dff-d84d-4f6a-b4de-c8eaa3500aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:39.609215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a2dff-d84d-4f6a-b4de-c8eaa3500aca service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:39.623806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1230/2490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:39.631197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:39.637959 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.650337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795978c6-f9d8-4467-9bc5-1052f9d6ca94 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:39.655874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:39.656561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.656846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.657125 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] PATH_INFO: `/v3/projects/f34ec21f18b14f59a716fe013a75df88` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.658656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f34ec21f18b14f59a716fe013a75df88)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:39.660747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:39.723568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34ec21f18b14f59a716fe013a75df88', 'request_id': 'req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.730643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34ec21f18b14f59a716fe013a75df88', 'request_id': 'req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.744498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34ec21f18b14f59a716fe013a75df88', 'request_id': 'req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.799120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] The token cache is being invalidate because project f34ec21f18b14f59a716fe013a75df88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:39.799120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.799120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-90fbc19b-f3b2-4634-8bc2-9c37230edf4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:39.799120 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1261/2491] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:39 2022] DELETE /identity/v3/projects/f34ec21f18b14f59a716fe013a75df88 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:39.806309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.806607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.807009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.831974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:39.833361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.849131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:39.921791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1231/2492] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:39.930862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.931439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.931721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.955060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:39.957171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:39.969862 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1262/2493] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:39.971778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:39.982102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94f6e-1edf-4c29-8d08-e3598eb11bbd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:39.992390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:39.993066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:39.993332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:39.993602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:39.994284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:41:39.994582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:39.996294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.016090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1232/2494] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:41:40.017732 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.050632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e1101-a6d9-40db-9700-ae18928b3079 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.050632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.050632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.050632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.051090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.051090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:40.052855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.090525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1263/2495] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:41:40 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:41:40.095881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.108338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b631a-1b81-4663-9367-00e4a394e7a9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.216706 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.217447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.217592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.217873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.218568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:40.221975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.272038 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1233/2496] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:41:40 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:41:40.272239 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.280814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e62ea-4211-42b3-8289-61571a91767b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.289574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.290906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.291223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.291485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.292330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:40.294287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.309483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1264/2497] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:41:40.310605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.324218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9d897-3107-4fb9-abef-884348f9a810 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.331186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e1ac3-eedd-4668-af88-340fcf94a4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.331861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1ac3-eedd-4668-af88-340fcf94a4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.332249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1ac3-eedd-4668-af88-340fcf94a4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.333046 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1ac3-eedd-4668-af88-340fcf94a4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.333814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e1ac3-eedd-4668-af88-340fcf94a4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:40.336471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e1ac3-eedd-4668-af88-340fcf94a4a8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.354019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1234/2498] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:41:40.355765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.368796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572c96-e7cd-469e-8882-8bb89edb139e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.375703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.377244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.377578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.377882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] PATH_INFO: `/v3/projects/46a90b2120394f82a05ed5943d17f1d0/users/ff9ec9bc2c9b4b33b81d2c9a35d3564b/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.428996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a90b2120394f82a05ed5943d17f1d0, user_id=ff9ec9bc2c9b4b33b81d2c9a35d3564b, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:40.439249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.461024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1265/2499] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:41:40 2022] PUT /identity/v3/projects/46a90b2120394f82a05ed5943d17f1d0/users/ff9ec9bc2c9b4b33b81d2c9a35d3564b/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:40.542410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.542693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.543298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.575382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] MFA Rules not processed for user `ff9ec9bc2c9b4b33b81d2c9a35d3564b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:40.576849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:40.614698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.617701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:40.707155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1235/2500] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:40.711045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:40.722790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:40.730243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:40.761896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4f5d6-1365-45e4-8134-560038cb18ba admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.763825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:40.769147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.769815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.770100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.770463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] PATH_INFO: `/v3/projects/79eb9f1de490416b81901b0421ef275c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.771695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79eb9f1de490416b81901b0421ef275c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:40.774817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:40.843130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79eb9f1de490416b81901b0421ef275c', 'request_id': 'req-92138205-0c83-48c1-bbfc-082481580cf5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:40.852417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79eb9f1de490416b81901b0421ef275c', 'request_id': 'req-92138205-0c83-48c1-bbfc-082481580cf5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:40.854057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0db10c-7c81-4012-aca9-addc3da60add None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:40.865113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79eb9f1de490416b81901b0421ef275c', 'request_id': 'req-92138205-0c83-48c1-bbfc-082481580cf5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:40.884379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] The token cache is being invalidate because project 79eb9f1de490416b81901b0421ef275c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:40.885945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:40.886854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-92138205-0c83-48c1-bbfc-082481580cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:40.888530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1266/2501] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:40 2022] DELETE /identity/v3/projects/79eb9f1de490416b81901b0421ef275c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:40.927321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:40.928023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:40.928325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:40.928630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:40.930624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:41.002957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:41.004425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:41.014775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1236/2502] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:44.884747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:44.885263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:44.885433 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:44.910408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:44.911615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:44.974448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1267/2503] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:44.988548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:44.990407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:45.097790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcd9c9-aec3-4102-8c30-be30dfefc135 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:45.103183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f7fa-3289-491f-ba5b-d49aab839125 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:45.103795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f7fa-3289-491f-ba5b-d49aab839125 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:45.104236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f7fa-3289-491f-ba5b-d49aab839125 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:45.104508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f7fa-3289-491f-ba5b-d49aab839125 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:45.115069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f7fa-3289-491f-ba5b-d49aab839125 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:45.118037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f7fa-3289-491f-ba5b-d49aab839125 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:45.127284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1237/2504] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:41:47.218443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:47.220891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:47.337110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bd98d-f8d3-4084-adba-1822be0ddde6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:47.367967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:47.371353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:47.372184 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:47.372584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] PATH_INFO: `/v3/users/ea40106c284c4492b2e6f8411d4372c8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:47.374923 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea40106c284c4492b2e6f8411d4372c8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:47.391022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:47.419409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:47.456989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea40106c284c4492b2e6f8411d4372c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:47.464435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea40106c284c4492b2e6f8411d4372c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:47.465136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea40106c284c4492b2e6f8411d4372c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:47.474209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea40106c284c4492b2e6f8411d4372c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:47.490090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea40106c284c4492b2e6f8411d4372c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:47.500090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1268/2505] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:47 2022] DELETE /identity/v3/users/ea40106c284c4492b2e6f8411d4372c8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:47.502784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:47.503460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:47.503460 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:47.525592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:47.527675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:47.536139 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:47.628270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1238/2506] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:47.648004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:47.648004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:47.729655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60741e1d-c231-4fce-ab56-81e147a649fd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:47.734177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:47.734989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:47.735455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:47.735873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:47.750307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:47.752961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:47.765794 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1269/2507] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:48.135368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:48.137112 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:48.176185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5acc6d-0a8c-489d-9833-7b1032aab100 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:48.180896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:48.181539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:48.181819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:48.182110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] PATH_INFO: `/v3/projects/297efa814ad849888e4587aa971bc067` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:48.183214 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=297efa814ad849888e4587aa971bc067)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:48.185171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:48.271746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297efa814ad849888e4587aa971bc067', 'request_id': 'req-806195da-51c7-42da-bebe-111e7a8c81dc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.292885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297efa814ad849888e4587aa971bc067', 'request_id': 'req-806195da-51c7-42da-bebe-111e7a8c81dc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.305344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297efa814ad849888e4587aa971bc067', 'request_id': 'req-806195da-51c7-42da-bebe-111e7a8c81dc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.343901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] The token cache is being invalidate because project 297efa814ad849888e4587aa971bc067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:48.344225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.344970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-806195da-51c7-42da-bebe-111e7a8c81dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.352050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1239/2508] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:48 2022] DELETE /identity/v3/projects/297efa814ad849888e4587aa971bc067 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:48.560339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:48.574258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:48.628265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d92d4b-7458-410a-bf37-4edf1fdb7f3d service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:48.764250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:48.764899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:48.765181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:48.765455 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] PATH_INFO: `/v3/users/ff9ec9bc2c9b4b33b81d2c9a35d3564b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:48.769373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9ec9bc2c9b4b33b81d2c9a35d3564b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:48.771829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:48.864414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:48.876782 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9ec9bc2c9b4b33b81d2c9a35d3564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.885599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9ec9bc2c9b4b33b81d2c9a35d3564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.886363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9ec9bc2c9b4b33b81d2c9a35d3564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.896169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9ec9bc2c9b4b33b81d2c9a35d3564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.918351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9ec9bc2c9b4b33b81d2c9a35d3564b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:48.931337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1270/2509] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:41:48 2022] DELETE /identity/v3/users/ff9ec9bc2c9b4b33b81d2c9a35d3564b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:48.936256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:48.937512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:48.937808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:48.981533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:48.983373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:49.109915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1240/2510] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:49.116997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:49.120823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:49.197563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592d25-d389-4637-9b9b-5d27866bf7ee admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:49.201158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a450055-79bb-4926-80aa-b63150b7a2f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:49.201761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a450055-79bb-4926-80aa-b63150b7a2f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:49.202204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a450055-79bb-4926-80aa-b63150b7a2f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:49.202414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a450055-79bb-4926-80aa-b63150b7a2f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:49.217101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a450055-79bb-4926-80aa-b63150b7a2f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:49.218798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a450055-79bb-4926-80aa-b63150b7a2f8 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:49.231139 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1271/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:41:49.614203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:49.614203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:49.631160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2120ce-cf2d-4260-8f06-8ff49d422cd8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:49.643344 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:49.644231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:49.644691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:49.645904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] PATH_INFO: `/v3/projects/46a90b2120394f82a05ed5943d17f1d0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:49.649952 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a90b2120394f82a05ed5943d17f1d0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:49.673313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:49.740427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:41:49.771207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a90b2120394f82a05ed5943d17f1d0', 'request_id': 'req-46c49382-78c2-4925-9989-5f7ec72c98a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:49.780514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a90b2120394f82a05ed5943d17f1d0', 'request_id': 'req-46c49382-78c2-4925-9989-5f7ec72c98a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:49.789268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a90b2120394f82a05ed5943d17f1d0', 'request_id': 'req-46c49382-78c2-4925-9989-5f7ec72c98a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:49.814676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] The token cache is being invalidate because project 46a90b2120394f82a05ed5943d17f1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:41:49.815049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:49.816292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:41:49.819005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1241/2512] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:41:49 2022] DELETE /identity/v3/projects/46a90b2120394f82a05ed5943d17f1d0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:41:57.097994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4491ab1-28ff-4e2a-b417-8fb5f9d1f655 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:57.098545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4491ab1-28ff-4e2a-b417-8fb5f9d1f655 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:57.098545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4491ab1-28ff-4e2a-b417-8fb5f9d1f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:57.119047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f4491ab1-28ff-4e2a-b417-8fb5f9d1f655 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:57.120298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f4491ab1-28ff-4e2a-b417-8fb5f9d1f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:57.184281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1272/2513] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:41:57.200286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:41:57.202566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:57.302476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c49382-78c2-4925-9989-5f7ec72c98a9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:41:57.307574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1764-525a-4ee0-8fbc-6b63cee14515 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:41:57.308209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1764-525a-4ee0-8fbc-6b63cee14515 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:57.308485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1764-525a-4ee0-8fbc-6b63cee14515 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:57.308754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1764-525a-4ee0-8fbc-6b63cee14515 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:57.318232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1764-525a-4ee0-8fbc-6b63cee14515 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:41:57.319863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1764-525a-4ee0-8fbc-6b63cee14515 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:41:57.328061 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1242/2514] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:41:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:41:58.156209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe558-c57b-48c0-afa1-aaf17e1f3d4c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:41:58.157360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe558-c57b-48c0-afa1-aaf17e1f3d4c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:41:58.157843 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe558-c57b-48c0-afa1-aaf17e1f3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:41:58.179687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-92bbe558-c57b-48c0-afa1-aaf17e1f3d4c None None] MFA Rules not processed for user `fded55a41aae41c6af61382a7c35372e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:41:58.181024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-92bbe558-c57b-48c0-afa1-aaf17e1f3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:41:58.254355 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1273/2515] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:01.046747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46627caa-ce7d-4ad3-b6c1-c2b92084321f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.047473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46627caa-ce7d-4ad3-b6c1-c2b92084321f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.048378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46627caa-ce7d-4ad3-b6c1-c2b92084321f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.049982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1243/2516] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:01.058353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.058866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.059030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.103152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:01.105644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.129395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:01.225529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1274/2517] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:01.239922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46627caa-ce7d-4ad3-b6c1-c2b92084321f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:01.242742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46627caa-ce7d-4ad3-b6c1-c2b92084321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.349280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46627caa-ce7d-4ad3-b6c1-c2b92084321f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:01.355794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9f552-c550-4531-9276-ebe0bb3d70a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:01.357129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9f552-c550-4531-9276-ebe0bb3d70a6 service glance] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.357597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9f552-c550-4531-9276-ebe0bb3d70a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.358082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9f552-c550-4531-9276-ebe0bb3d70a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.374849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9f552-c550-4531-9276-ebe0bb3d70a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:01.378717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9f552-c550-4531-9276-ebe0bb3d70a6 service glance] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:01.404974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1244/2518] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:01.466286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:01.468020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.626956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870b46e-f30e-45e2-84c1-e5decc1af2e7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.630096 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870b46e-f30e-45e2-84c1-e5decc1af2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.630710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870b46e-f30e-45e2-84c1-e5decc1af2e7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.633922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1245/2519] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:42:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:01.641183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.641468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.641835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.728468 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:01.729794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.768160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a697787-16d6-4675-a96f-47cc7e65af55 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:01.778489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:01.788580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.788903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.789322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] PATH_INFO: `/v3/users/f853046d4e574976a74eaf0838097f16` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.793463 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f853046d4e574976a74eaf0838097f16)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:01.799075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:01.870334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:01.883166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1246/2520] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:42:01.900204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:01.901923 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.913106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f853046d4e574976a74eaf0838097f16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:01.916698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b13816-eecc-481c-a4ff-fb030a7dc936 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:01.921070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f853046d4e574976a74eaf0838097f16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:01.921249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:01.921957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.922375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.922712 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:01.922888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f853046d4e574976a74eaf0838097f16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:01.924579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:01.936192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f853046d4e574976a74eaf0838097f16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:01.939024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:01.940687 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7908897-822a-4384-a6ff-aa427b327dc7 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:01.951653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1247/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:01.963173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8cc93cf7-7994-4f33-b814-19215c9cae13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f853046d4e574976a74eaf0838097f16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:01.974766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1275/2522] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:42:01 2022] DELETE /identity/v3/users/f853046d4e574976a74eaf0838097f16 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:01.979662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:01.980057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:01.980446 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:02.004675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:02.006050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:02.079327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1248/2523] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:02.097152 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a67b3-821e-4b13-83c9-f2d1a88cd9c4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:02.097861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a67b3-821e-4b13-83c9-f2d1a88cd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:02.097861 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a67b3-821e-4b13-83c9-f2d1a88cd9c4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:02.098274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:02.099052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1276/2524] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:02.105163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:02.108634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5246447-3fee-4283-9637-4f591058810c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:02.109023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5246447-3fee-4283-9637-4f591058810c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:02.109359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5246447-3fee-4283-9637-4f591058810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:02.135997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c5246447-3fee-4283-9637-4f591058810c None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:02.137531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c5246447-3fee-4283-9637-4f591058810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:02.158822 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:02.221057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ca925-75bb-49fc-8764-e34e5597d6ad None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:02.227409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:02.228445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:02.229077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:02.229572 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:02.239481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1277/2525] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:02.241790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:02.243803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:02.249663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5246447-3fee-4283-9637-4f591058810c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:02.258265 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1249/2526] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:42:02.262913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5246447-3fee-4283-9637-4f591058810c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:02.267554 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d365e-7b16-46d7-8193-01c77ed36c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:02.268123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d365e-7b16-46d7-8193-01c77ed36c6c service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:02.268426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d365e-7b16-46d7-8193-01c77ed36c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:02.268715 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d365e-7b16-46d7-8193-01c77ed36c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:02.284520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d365e-7b16-46d7-8193-01c77ed36c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:02.286122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d365e-7b16-46d7-8193-01c77ed36c6c service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:02.300651 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1278/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:02.897435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:02.899206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:02.946470 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb24100-8389-430d-856d-25e13e61477a service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:02.960514 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:02.961379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:02.961766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:02.962150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] PATH_INFO: `/v3/projects/141f2ca8eee3453bad5422d3f6169e5e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:02.964209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141f2ca8eee3453bad5422d3f6169e5e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:02.966533 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.066996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141f2ca8eee3453bad5422d3f6169e5e', 'request_id': 'req-e36062f3-5449-4e6c-80c0-512307360a19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:03.075317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141f2ca8eee3453bad5422d3f6169e5e', 'request_id': 'req-e36062f3-5449-4e6c-80c0-512307360a19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:03.113598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141f2ca8eee3453bad5422d3f6169e5e', 'request_id': 'req-e36062f3-5449-4e6c-80c0-512307360a19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:03.156176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] The token cache is being invalidate because project 141f2ca8eee3453bad5422d3f6169e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:42:03.156770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:03.157945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e36062f3-5449-4e6c-80c0-512307360a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:03.166732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1250/2528] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:42:02 2022] DELETE /identity/v3/projects/141f2ca8eee3453bad5422d3f6169e5e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:03.171709 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a551af-6362-4743-a879-169eca5956d9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.174185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a551af-6362-4743-a879-169eca5956d9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.174564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a551af-6362-4743-a879-169eca5956d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.224022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-34a551af-6362-4743-a879-169eca5956d9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:03.226167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-34a551af-6362-4743-a879-169eca5956d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:03.335607 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1279/2529] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:03.338622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.338622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.338622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.362790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:03.364419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:03.381124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1251/2530] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:03.394357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a551af-6362-4743-a879-169eca5956d9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:03.409160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a551af-6362-4743-a879-169eca5956d9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:03.415414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:03.416200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.416586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.417028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.417812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:42:03.418307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:03.420204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.436680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1280/2531] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:42:03.443702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:03.455843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1c8ce-7280-4854-8eae-e521858fb2f4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:03.461604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:03.464238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.465080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.465080 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.467487 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:03.470400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.477316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:03.521802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1252/2532] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:42:03 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:42:03.527392 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:03.540438 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257109e5-b122-4d48-bc56-8b1a811fdf92 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:03.643357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:03.650140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.650568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.651998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.653315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:03.658218 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.685567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1281/2533] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:42:03 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:03.690012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:03.708773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6eea-452d-467c-849a-3a6aef9d871f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:03.718184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:03.720134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.720565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.721123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.722165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:03.724437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.740003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1253/2534] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:42:03.741290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:03.764552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbcf4-c426-42ed-965a-5844b8da1d83 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:03.776008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea6794-05d6-4955-afd0-4c0128c81bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:03.778388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea6794-05d6-4955-afd0-4c0128c81bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.778714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea6794-05d6-4955-afd0-4c0128c81bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.778998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea6794-05d6-4955-afd0-4c0128c81bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.779806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea6794-05d6-4955-afd0-4c0128c81bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:03.783646 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea6794-05d6-4955-afd0-4c0128c81bb3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.807785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:03.808177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1282/2535] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:03.819753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a95c79-efb7-41ac-9a83-d5c2810ad3f0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:03.826011 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:03.826864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.827254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.827653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] PATH_INFO: `/v3/projects/cd41122fd4e94aa786798d8f54214f0a/users/9fb81fa5765c44a0afa8d9fb4786272c/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.853634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd41122fd4e94aa786798d8f54214f0a, user_id=9fb81fa5765c44a0afa8d9fb4786272c, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:03.856640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:03.878902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1254/2536] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:03 2022] PUT /identity/v3/projects/cd41122fd4e94aa786798d8f54214f0a/users/9fb81fa5765c44a0afa8d9fb4786272c/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:03.885705 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:03.886091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:03.886479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:03.905836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:03.916306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:03.918464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:04.085246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1283/2537] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:42:04.095249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:04.097500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:04.204840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234afc2-80ba-4c54-ad88-0da71100db3b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:04.210558 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89290f8-510a-4197-995b-494d37ef132f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:04.211416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89290f8-510a-4197-995b-494d37ef132f service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:04.211901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89290f8-510a-4197-995b-494d37ef132f service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:04.212433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89290f8-510a-4197-995b-494d37ef132f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:04.231312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89290f8-510a-4197-995b-494d37ef132f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:04.233135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89290f8-510a-4197-995b-494d37ef132f service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:04.248892 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1255/2538] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:06.827217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:06.828752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:06.840992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c94568-fae1-4fee-baee-83dcce3713bd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:06.849442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:06.850332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:06.850792 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:06.851234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] PATH_INFO: `/v3/users/17d6b047899a46d7b6ce1292bf5792ea` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:06.852779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d6b047899a46d7b6ce1292bf5792ea)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:06.854863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:06.928611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d6b047899a46d7b6ce1292bf5792ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:06.947648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d6b047899a46d7b6ce1292bf5792ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:06.948512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d6b047899a46d7b6ce1292bf5792ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:06.963123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d6b047899a46d7b6ce1292bf5792ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:06.990982 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d6b047899a46d7b6ce1292bf5792ea'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:07.038470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1284/2539] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:42:06 2022] DELETE /identity/v3/users/17d6b047899a46d7b6ce1292bf5792ea => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:07.042825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ec893-056e-4951-a6f4-0fa8f1630c6a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:07.043694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ec893-056e-4951-a6f4-0fa8f1630c6a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:07.044165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ec893-056e-4951-a6f4-0fa8f1630c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:07.080395 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2e2ec893-056e-4951-a6f4-0fa8f1630c6a None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:07.081849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2e2ec893-056e-4951-a6f4-0fa8f1630c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.181078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1256/2540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:07.190272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:07.191846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.253192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:07.254301 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:07.254684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:07.319277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:07.324278 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.349023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5c00d-eb8a-4fef-8d1b-7abe15857cac admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:07.355458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:07.360734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:07.361057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:07.361342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:07.385115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:07.387319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:07.417813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1285/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:07.421281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:07.436120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1257/2542] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:07.460354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:07.463094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:07.503234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:07.638346 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7e1f-f7ad-4cf0-bcf7-b8db22179421 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:07.657468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ed6ca-5792-43db-9b2a-348d2b04a1e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:07.659583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ed6ca-5792-43db-9b2a-348d2b04a1e8 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:07.660347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ed6ca-5792-43db-9b2a-348d2b04a1e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:07.660740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ed6ca-5792-43db-9b2a-348d2b04a1e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:07.682237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ed6ca-5792-43db-9b2a-348d2b04a1e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:07.683857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ed6ca-5792-43db-9b2a-348d2b04a1e8 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:07.713585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1286/2543] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:42:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:42:08.579617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:08.582079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:08.611495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50168-27b0-4fd4-985c-2fa17ad1e5d4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:08.628550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:08.629351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:08.629713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:08.630055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] PATH_INFO: `/v3/projects/9c4a2e2df8844081a447f008d28ea006` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:08.633611 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4a2e2df8844081a447f008d28ea006)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:08.635868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:08.829613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4a2e2df8844081a447f008d28ea006', 'request_id': 'req-362962fd-b7e0-41f9-ad46-429ac7e20058'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:08.879700 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4a2e2df8844081a447f008d28ea006', 'request_id': 'req-362962fd-b7e0-41f9-ad46-429ac7e20058'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:08.896899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4a2e2df8844081a447f008d28ea006', 'request_id': 'req-362962fd-b7e0-41f9-ad46-429ac7e20058'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:08.920260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] The token cache is being invalidate because project 9c4a2e2df8844081a447f008d28ea006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:42:08.920730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:08.924057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-362962fd-b7e0-41f9-ad46-429ac7e20058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:08.926354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1258/2544] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:42:08 2022] DELETE /identity/v3/projects/9c4a2e2df8844081a447f008d28ea006 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:08.932136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:08.932638 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:08.932718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:08.970718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:08.972219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:09.128079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1287/2545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:09.128924 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.129218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.129521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.279043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:09.279504 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:09.311310 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1259/2546] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:09.342563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:09.394255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a82300-e587-4eb5-ba62-ee2fc17c2bfe None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:09.409778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:09.410410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.410692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.410995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.411622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:42:09.411924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:09.422408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:09.468057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1288/2547] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:42:09.474493 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:09.487505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6185-1608-452b-8c22-1b5e2de35656 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:09.504219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:09.506209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.506494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.509429 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.529471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:09.540449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:09.623979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1260/2548] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:42:09 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:09.625505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:09.648004 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6f318-573d-4def-8717-3fb2ca9f2c24 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:09.833405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:09.834037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.834425 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.834714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.835431 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:09.837520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:09.895476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:09.909512 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1289/2549] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:42:09 2022] POST /identity/v3/users => generated 290 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:09.931509 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a95626-07c7-440d-8e5d-19b65ee5b955 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:09.951988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:09.952627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:09.953015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:09.953401 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:09.954497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:09.956191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:09.988316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:10.009623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1261/2550] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:42:10.025724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9131f1-d28b-4c92-85bb-55262d16e4c6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:10.058812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85793f66-0752-4f85-99be-00edc97a5b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:10.059118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793f66-0752-4f85-99be-00edc97a5b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:10.059330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793f66-0752-4f85-99be-00edc97a5b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:10.059605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85793f66-0752-4f85-99be-00edc97a5b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:10.060285 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793f66-0752-4f85-99be-00edc97a5b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:10.061887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85793f66-0752-4f85-99be-00edc97a5b06 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:10.087496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1290/2551] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:10.094696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:10.109176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3545e-ea12-41de-9852-61adb6c8e634 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:10.118718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:10.119411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:10.119690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:10.119984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] PATH_INFO: `/v3/projects/7c99de6ca9094fa7bd119a3be6817725/users/bb5d3834f04142e288c04cc05665329f/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:10.144366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c99de6ca9094fa7bd119a3be6817725, user_id=bb5d3834f04142e288c04cc05665329f, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:10.155521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:10.199300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1262/2552] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:10 2022] PUT /identity/v3/projects/7c99de6ca9094fa7bd119a3be6817725/users/bb5d3834f04142e288c04cc05665329f/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:42:10.207440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7424-1d57-4eda-bbab-64f4ebbf35d8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:10.207801 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7424-1d57-4eda-bbab-64f4ebbf35d8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:10.208227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7424-1d57-4eda-bbab-64f4ebbf35d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:10.270316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3a2b7424-1d57-4eda-bbab-64f4ebbf35d8 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:10.276777 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a2b7424-1d57-4eda-bbab-64f4ebbf35d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:10.322529 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:10.400877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1291/2553] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:10.425373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:10.428189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:10.528189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e0ff3-b04c-4848-8d57-8e5cca7184d8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:10.533202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:10.533904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:10.534587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:10.534587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:10.551077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:10.553565 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:10.580999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1263/2554] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:13.704364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade3fa-6985-4755-be50-950929711146 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:13.704884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade3fa-6985-4755-be50-950929711146 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:13.705117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade3fa-6985-4755-be50-950929711146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:13.763102 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-43ade3fa-6985-4755-be50-950929711146 None None] MFA Rules not processed for user `9fb81fa5765c44a0afa8d9fb4786272c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:13.764591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-43ade3fa-6985-4755-be50-950929711146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:13.924374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1292/2555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:13.925635 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:13.927294 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:13.960279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e525-b84d-4806-b6e1-86e1c5c2f0db service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:13.981684 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:13.983990 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:13.986128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:13.986599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:13.996113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.002302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.072928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:14.083129 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1264/2556] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:42:13 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:14.088765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade3fa-6985-4755-be50-950929711146 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:14.105602 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade3fa-6985-4755-be50-950929711146 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:14.266604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:14.266604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.268180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.268571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.269365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.272656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.325106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1293/2557] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:42:14 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:14.329779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:14.344191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063165d-f97b-4370-a7b8-8cd5f7c61b28 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:14.362335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:14.363197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.363738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.364145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.364931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.368668 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.398897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1265/2558] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:14.403464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:14.421112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0393cec-61de-4838-a721-c40952e44e4e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:14.436831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:14.441394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.441394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.441394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] PATH_INFO: `/v3/projects/01e3335bc64744e1b2ee2fe5c061f881/users/5dd839a82aae4ce4a94fee87a85a1b34/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.480079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e3335bc64744e1b2ee2fe5c061f881, user_id=5dd839a82aae4ce4a94fee87a85a1b34, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.483422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.521287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1294/2559] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:14 2022] PUT /identity/v3/projects/01e3335bc64744e1b2ee2fe5c061f881/users/5dd839a82aae4ce4a94fee87a85a1b34/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:14.524986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:14.541570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40782c-9d66-44c1-a23a-0ac135c2b836 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:14.686642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2fd2f-ffe8-4201-baa2-234022bdf710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:14.687799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2fd2f-ffe8-4201-baa2-234022bdf710 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.688114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2fd2f-ffe8-4201-baa2-234022bdf710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.688456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2fd2f-ffe8-4201-baa2-234022bdf710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.689184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2fd2f-ffe8-4201-baa2-234022bdf710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.691199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2fd2f-ffe8-4201-baa2-234022bdf710 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.723132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1266/2560] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:14.727894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:14.744201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472462e-93be-4c4e-8ba3-3d09187e3f0a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:14.756305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:14.757422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.757771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.758174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] PATH_INFO: `/v3/projects/01e3335bc64744e1b2ee2fe5c061f881/users/5dd839a82aae4ce4a94fee87a85a1b34/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.761557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e3335bc64744e1b2ee2fe5c061f881, user_id=5dd839a82aae4ce4a94fee87a85a1b34, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.769556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.786768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:14.797503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1295/2561] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:14 2022] PUT /identity/v3/projects/01e3335bc64744e1b2ee2fe5c061f881/users/5dd839a82aae4ce4a94fee87a85a1b34/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:42:24.712794 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=35 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:24.761016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:24.761336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:24.761585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:24.811239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:24.813551 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:24.993432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1267/2562] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 16:42:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:25.045578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:25.047547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:25.084973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:25.086735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:25.241407 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc07b8-36aa-41c7-a53a-1b106aaa0bd8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:25.271379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8822a2-e30a-44a4-9af4-533251a0fe71 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:25.271864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65fa42-8c66-40de-98a4-a0d3ba85c121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:25.272824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65fa42-8c66-40de-98a4-a0d3ba85c121 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:25.273270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65fa42-8c66-40de-98a4-a0d3ba85c121 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:25.273710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65fa42-8c66-40de-98a4-a0d3ba85c121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:25.290111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:25.290915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:25.291300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:25.291686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:25.294116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:25.299151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65fa42-8c66-40de-98a4-a0d3ba85c121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:25.301149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65fa42-8c66-40de-98a4-a0d3ba85c121 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:25.322817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:25.324690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:25.329754 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1296/2563] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:42:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:42:25.349284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1268/2564] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:42:27.757042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:27.757670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:27.758162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:27.871147 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] MFA Rules not processed for user `bb5d3834f04142e288c04cc05665329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:27.872740 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:28.185325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1297/2565] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:28.186013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:28.407915 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32751f-5b59-4de3-b1ef-236ad398cb2e service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:28.426534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:28.427179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:28.427508 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:28.427726 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:28.428407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:28.430336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:28.532271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:28.535682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1269/2566] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:42:28 2022] POST /identity/v3/projects => generated 379 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:28.554713 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c4a07-e0a0-4ad7-a2b6-8b672c228c68 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:28.837129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:28.861676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:28.863333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:28.863333 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:28.865462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:28.876291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:28.941987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:28.954610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1298/2567] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:42:28 2022] POST /identity/v3/users => generated 296 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:42:28.962783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a21191-80a9-461e-9a48-1946cfe60b5d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:28.972741 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001026c7-848a-4972-8afb-13356be78a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:28.973497 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001026c7-848a-4972-8afb-13356be78a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:28.973783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001026c7-848a-4972-8afb-13356be78a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:28.974816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001026c7-848a-4972-8afb-13356be78a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:28.975375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001026c7-848a-4972-8afb-13356be78a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:28.978941 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001026c7-848a-4972-8afb-13356be78a1c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:29.026408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1270/2568] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:29.030578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:29.057864 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab5d55-9033-4422-b92a-43c1a7b8882f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:29.060155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:29.060441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:29.070097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:29.078434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:29.079305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:29.079593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:29.079875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] PATH_INFO: `/v3/projects/2bf4dc9d2b8140ba8de7cf0b15d5fd63/users/f7aad9b95e804d4a93ae6975ee52b39b/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:29.090503 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:29.175251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:29.190727 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf4dc9d2b8140ba8de7cf0b15d5fd63, user_id=f7aad9b95e804d4a93ae6975ee52b39b, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:29.198036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:29.208374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] MFA Rules not processed for user `5dd839a82aae4ce4a94fee87a85a1b34`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:29.211801 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:29.239887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1299/2569] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:29 2022] PUT /identity/v3/projects/2bf4dc9d2b8140ba8de7cf0b15d5fd63/users/f7aad9b95e804d4a93ae6975ee52b39b/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:42:29.240920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:29.349764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2c33-0750-4493-af04-3c0fffca461f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:29.355847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:29.356535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:29.356819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:29.357105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:29.357766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:29.360270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:29.376843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1271/2570] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:29.388675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1300/2571] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:29.396195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:29.396579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:29.398097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:29.418203 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d021f-739a-45bc-ab0e-7ef0e93810c8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:29.426708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:29.428445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:29.428854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:29.429252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] PATH_INFO: `/v3/projects/2bf4dc9d2b8140ba8de7cf0b15d5fd63/users/f7aad9b95e804d4a93ae6975ee52b39b/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:29.431879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf4dc9d2b8140ba8de7cf0b15d5fd63, user_id=f7aad9b95e804d4a93ae6975ee52b39b, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:29.436490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:29.456277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1272/2572] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:29 2022] PUT /identity/v3/projects/2bf4dc9d2b8140ba8de7cf0b15d5fd63/users/f7aad9b95e804d4a93ae6975ee52b39b/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:29.557450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be521-75fe-4aad-870d-c3856b925417 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:29.724675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:29.725574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:29.726013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:29.726446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:29.759421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:29.812699 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:29.814584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:29.884380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:29.886014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:29.934699 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1301/2573] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:29.963507 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65378d9-9964-42c4-a3c8-abc111978c74 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:29.975079 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:29.975754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:29.978407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:29.978984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] PATH_INFO: `/v3/users/e4b36e6959444273872c300742d6ddb8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:29.980879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4b36e6959444273872c300742d6ddb8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:29.986521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:30.114587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4b36e6959444273872c300742d6ddb8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.143790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4b36e6959444273872c300742d6ddb8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.166421 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4b36e6959444273872c300742d6ddb8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.172350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4b36e6959444273872c300742d6ddb8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.189175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a358c70b-7f31-4ce4-9cc3-0e431df35f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4b36e6959444273872c300742d6ddb8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.192173 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:30.209765 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1273/2574] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:42:29 2022] DELETE /identity/v3/users/e4b36e6959444273872c300742d6ddb8 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:30.528117 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:30.544177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2389d-eff0-4d0c-9db4-ac4ceac44eb7 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:30.703133 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:30.703984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:30.704363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:30.704643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] PATH_INFO: `/v3/projects/8424741e1d154c6c9b08b19fb7c61d33` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:30.705866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8424741e1d154c6c9b08b19fb7c61d33)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:30.708054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:30.794841 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8424741e1d154c6c9b08b19fb7c61d33', 'request_id': 'req-30265a15-70bc-4761-b4bc-2bf85bda1362'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.804297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8424741e1d154c6c9b08b19fb7c61d33', 'request_id': 'req-30265a15-70bc-4761-b4bc-2bf85bda1362'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.826813 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8424741e1d154c6c9b08b19fb7c61d33', 'request_id': 'req-30265a15-70bc-4761-b4bc-2bf85bda1362'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.844895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] The token cache is being invalidate because project 8424741e1d154c6c9b08b19fb7c61d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:42:30.846095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.847164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-30265a15-70bc-4761-b4bc-2bf85bda1362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:42:30.850037 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1302/2575] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:42:30 2022] DELETE /identity/v3/projects/8424741e1d154c6c9b08b19fb7c61d33 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:42:30.856260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:30.856640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:30.857000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:30.902738 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:30.904552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:31.044394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1274/2576] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:31.053020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.053537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.053772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.076664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:31.078536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:31.101363 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1303/2577] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:31.105933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.120001 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa53044-576a-44ab-971e-03b2ef5376d3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.127598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.128451 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.128799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.129135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.129916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:42:31.130521 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.134013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.153374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1275/2578] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:42:31.158007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.170957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75053b-3e52-4174-b440-cf1c15d2b34e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.177416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.178915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.179889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.180417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.181326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.183647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.230721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1304/2579] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:42:31 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:31.235435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.250983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9fd03-b2b0-4c2d-9daf-6f550989dbc9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.340793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.341415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.341701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.342100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.342770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.345246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.372640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1276/2580] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:42:31 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:42:31.377759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.397335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf427e-7151-4cdd-975b-1d2b2c718a77 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.404024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.404771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.405118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.405608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.406759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.408776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.425752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1305/2581] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:42:31.429428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.440919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee52c4-1bc1-4f4b-a8bd-124af98f7eec admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.448927 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537fcfa-786d-400f-93fc-ded493b9bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.451174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537fcfa-786d-400f-93fc-ded493b9bfae admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.451594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537fcfa-786d-400f-93fc-ded493b9bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.452024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537fcfa-786d-400f-93fc-ded493b9bfae admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.453845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537fcfa-786d-400f-93fc-ded493b9bfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.456068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537fcfa-786d-400f-93fc-ded493b9bfae admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.481293 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.482462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1277/2582] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:42:31.494831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa4d7c-419d-451c-aa88-f507372b63ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.501426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.502724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.503364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.503909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] PATH_INFO: `/v3/projects/099d0dfa51be462e9650ab8ce50465fb/users/ad81d5c075d64fa8978122e124d00e19/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.509825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:31.529675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099d0dfa51be462e9650ab8ce50465fb, user_id=ad81d5c075d64fa8978122e124d00e19, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.536129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.561421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1306/2583] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:42:31 2022] PUT /identity/v3/projects/099d0dfa51be462e9650ab8ce50465fb/users/ad81d5c075d64fa8978122e124d00e19/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:42:31.564251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98ccee-0428-4e85-b2ff-2015671df5f7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.564538 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98ccee-0428-4e85-b2ff-2015671df5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.564817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98ccee-0428-4e85-b2ff-2015671df5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.594669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ea98ccee-0428-4e85-b2ff-2015671df5f7 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:31.595975 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ea98ccee-0428-4e85-b2ff-2015671df5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:31.684239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1278/2584] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:31.695210 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:31.697241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:31.789679 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17006cd9-2988-4d16-9946-17675b7d30c3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:31.794568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541526d2-15e2-49db-b9d1-fd518ad0b3d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:31.796196 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541526d2-15e2-49db-b9d1-fd518ad0b3d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:31.796552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541526d2-15e2-49db-b9d1-fd518ad0b3d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:31.797852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541526d2-15e2-49db-b9d1-fd518ad0b3d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:31.814800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541526d2-15e2-49db-b9d1-fd518ad0b3d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:31.814800 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541526d2-15e2-49db-b9d1-fd518ad0b3d9 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:31.842442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1307/2585] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:32.678518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:32.678988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:32.679292 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:32.705977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:32.717161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] MFA Rules not processed for user `9fb81fa5765c44a0afa8d9fb4786272c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:32.722415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:32.855049 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1279/2586] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:32.926977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fc0d1-d6ad-441a-91cf-036ac72bae64 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:32.928976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fc0d1-d6ad-441a-91cf-036ac72bae64 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:32.929426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fc0d1-d6ad-441a-91cf-036ac72bae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:32.977920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-583fc0d1-d6ad-441a-91cf-036ac72bae64 None None] MFA Rules not processed for user `ad81d5c075d64fa8978122e124d00e19`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:32.979997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-583fc0d1-d6ad-441a-91cf-036ac72bae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:33.074000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1308/2587] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:33.088516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:33.089968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:33.166208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36571ca-54b5-45d0-8323-2d8aa2eb2c2a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:33.171672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd3c7-1826-4167-8d1d-3399e8b5f2c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:33.172406 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd3c7-1826-4167-8d1d-3399e8b5f2c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:33.172686 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd3c7-1826-4167-8d1d-3399e8b5f2c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:33.173588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd3c7-1826-4167-8d1d-3399e8b5f2c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:33.183342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd3c7-1826-4167-8d1d-3399e8b5f2c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:33.184946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd3c7-1826-4167-8d1d-3399e8b5f2c5 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:33.197824 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1280/2588] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:34.961226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:34.962750 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:34.963305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:34.997158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] MFA Rules not processed for user `ad81d5c075d64fa8978122e124d00e19`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:34.998734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:35.008519 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1309/2589] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:36.816956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d532-f2b7-4a89-aa9e-47487bdfeb31 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:36.818044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d532-f2b7-4a89-aa9e-47487bdfeb31 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:36.818792 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d532-f2b7-4a89-aa9e-47487bdfeb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:36.879989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-82a8d532-f2b7-4a89-aa9e-47487bdfeb31 None None] MFA Rules not processed for user `f7aad9b95e804d4a93ae6975ee52b39b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:36.881380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-82a8d532-f2b7-4a89-aa9e-47487bdfeb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:37.060368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1281/2590] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:42:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:37.068136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:37.069644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:37.168095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf294189-fb50-49e4-bccf-afdc9da908d4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:37.175237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:37.176042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:37.177435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:37.177721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:37.179893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:37.191021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:37.263465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:37.266320 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:37.283121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1310/2591] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:44.907733 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565898dc-dafb-404a-9092-9189f512ae7e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:44.908309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565898dc-dafb-404a-9092-9189f512ae7e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:44.908403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565898dc-dafb-404a-9092-9189f512ae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:44.931605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-565898dc-dafb-404a-9092-9189f512ae7e None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:44.933660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-565898dc-dafb-404a-9092-9189f512ae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:44.978026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:45.007870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1282/2592] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 16:42:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:45.025729 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:45.027515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:45.127657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cde69-d733-45c3-b385-563341880049 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:45.132812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:45.133413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:45.133741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:45.133977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:45.143719 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:45.145641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:45.156894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1311/2593] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:42:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:42:55.010407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:55.151167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=57 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:43:01.601510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:43:01.602059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:43:01.602059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:43:01.621051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] MFA Rules not processed for user `bb5d3834f04142e288c04cc05665329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:43:01.622280 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:43:01.629628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1283/2594] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:43:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:43:08.808556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:43:08.811480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:43:08.828511 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eceff8-8bc1-4033-b3d9-88c5e1104352 service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:43:08.832916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:43:08.833610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:43:08.833823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:43:08.834106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:43:08.835908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:43:08.934488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:43:08.938809 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51063475-b48b-4de2-bc51-c4f6c19419a9 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:43:08.961489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1312/2595] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:43:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:43:11.615969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:43:18.951094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=30 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:43:26.548520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:43:26.561163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f08e0-486a-4115-aeeb-d780b00c3948 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:43:26.564902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1f398-405d-4119-bf6e-abcbc83d2f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:43:26.565505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f398-405d-4119-bf6e-abcbc83d2f4c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:43:26.565776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f398-405d-4119-bf6e-abcbc83d2f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:43:26.566236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f398-405d-4119-bf6e-abcbc83d2f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:43:26.577644 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1f398-405d-4119-bf6e-abcbc83d2f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:43:26.579321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1f398-405d-4119-bf6e-abcbc83d2f4c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:43:26.589878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1284/2596] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:43:34.376586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28648cd3-973e-4596-9c83-bb6a09de2b79 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:43:34.377090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28648cd3-973e-4596-9c83-bb6a09de2b79 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:43:34.377189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28648cd3-973e-4596-9c83-bb6a09de2b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:43:34.396907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-28648cd3-973e-4596-9c83-bb6a09de2b79 None None] MFA Rules not processed for user `fded55a41aae41c6af61382a7c35372e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:43:34.398121 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-28648cd3-973e-4596-9c83-bb6a09de2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:43:34.454896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1313/2597] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:43:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:43:36.585547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:43:44.447606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=25 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:43:49.507268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:43:49.507806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:43:49.507806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:43:49.527530 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] MFA Rules not processed for user `ad81d5c075d64fa8978122e124d00e19`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:43:49.528867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:43:49.535865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1285/2598] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:43:56.949685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:43:56.950191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:43:56.950303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:43:56.969472 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] MFA Rules not processed for user `79d593b826c64087bac576d2204302b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:43:56.970836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:43:57.025771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1314/2599] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:43:59.521841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:07.018868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=25 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:23.273857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:23.275296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:23.288115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9c99-ff97-4711-8414-48344dcce631 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:23.300307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:23.300946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.301228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.301506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.303744 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:23.318433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:23.319878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:23.331246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1286/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:25.965984 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.976509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693c6f6-0812-4aa5-a136-6a0be90db1ba None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:25.980064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:25.980659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.980935 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.981206 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.982960 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:25.996110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.997520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:26.008956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1315/2601] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:29.710957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:29.723492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e78d2-4cf2-4ef7-b2f3-d5f495b9e965 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:29.729048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:29.729642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.729917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.730186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.731903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:29.743069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.744671 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.754869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1287/2602] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:36.003845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:39.749543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=12 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:49.630903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:49.643303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf36c7-ae4f-4f42-95d7-cfa58e9e51f5 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:49.650238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:49.650907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.651259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.651542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.653663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:49.668235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:49.670633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:49.688736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1316/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:50.614149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.623525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d47bf-ff9b-434e-babb-6466f0b8908b service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.627860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.628489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.628766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.629037 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.630780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:50.643709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.645208 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.654982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1288/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:51.171192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.172835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.233766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.246950 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea3a45-853b-453c-8d38-6aae43f49a50 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.252899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.253569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.253861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.254122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] PATH_INFO: `/v3/users/ad81d5c075d64fa8978122e124d00e19` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.255264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad81d5c075d64fa8978122e124d00e19)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.257438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.294315 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aae3e8-769c-48ee-a5ad-a468cdf3e1f2 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.301464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.301976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.302253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.302530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.304426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.321917 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad81d5c075d64fa8978122e124d00e19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.323043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.325115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.336758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad81d5c075d64fa8978122e124d00e19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.339177 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1317/2605] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:51.359501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad81d5c075d64fa8978122e124d00e19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.360266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad81d5c075d64fa8978122e124d00e19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.368413 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad81d5c075d64fa8978122e124d00e19'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.378276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1289/2606] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:51 2022] DELETE /identity/v3/users/ad81d5c075d64fa8978122e124d00e19 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:51.569815 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.586876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985c4dc-436e-4d5f-b881-b8bcf28905b3 service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.703156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.704906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.705422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.705866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] PATH_INFO: `/v3/projects/099d0dfa51be462e9650ab8ce50465fb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.707416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=099d0dfa51be462e9650ab8ce50465fb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.710006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.764857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099d0dfa51be462e9650ab8ce50465fb', 'request_id': 'req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.772974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099d0dfa51be462e9650ab8ce50465fb', 'request_id': 'req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.783697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099d0dfa51be462e9650ab8ce50465fb', 'request_id': 'req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.796090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] The token cache is being invalidate because project 099d0dfa51be462e9650ab8ce50465fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:51.796387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.797079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.799304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1318/2607] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:51 2022] DELETE /identity/v3/projects/099d0dfa51be462e9650ab8ce50465fb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:54.154405 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.156004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.213262 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.214696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.225944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:54.242153 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228cb33-79eb-4946-90cd-6b765f32574d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.289321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b23f0-63d1-4031-bbcc-37d904b4ad0e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.294043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.294678 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.294957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.295235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] PATH_INFO: `/v3/users/fded55a41aae41c6af61382a7c35372e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.296399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fded55a41aae41c6af61382a7c35372e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.299430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.304574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:54.312991 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.314503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.314977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.315411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.317501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.329625 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.331368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.341964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1290/2608] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:54.349974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fded55a41aae41c6af61382a7c35372e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.356925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fded55a41aae41c6af61382a7c35372e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.357606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fded55a41aae41c6af61382a7c35372e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.367252 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fded55a41aae41c6af61382a7c35372e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.380823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0397c0dd-61a5-4b04-9126-d0b3098b44a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fded55a41aae41c6af61382a7c35372e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.387851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1319/2609] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:54 2022] DELETE /identity/v3/users/fded55a41aae41c6af61382a7c35372e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:54.566373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.578302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54115e-7705-480f-b010-92b0829c8575 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.638591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.639219 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.639502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.639783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] PATH_INFO: `/v3/projects/364a00263e804efe83ae759d9c4c01fb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.641004 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=364a00263e804efe83ae759d9c4c01fb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.642982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.689128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364a00263e804efe83ae759d9c4c01fb', 'request_id': 'req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.695217 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364a00263e804efe83ae759d9c4c01fb', 'request_id': 'req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.701020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364a00263e804efe83ae759d9c4c01fb', 'request_id': 'req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.712999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] The token cache is being invalidate because project 364a00263e804efe83ae759d9c4c01fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:54.713290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.713913 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-63e4daf5-d0a6-4cb7-978d-5eb04c31e418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.715420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1291/2610] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:54 2022] DELETE /identity/v3/projects/364a00263e804efe83ae759d9c4c01fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:54.721324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.721613 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.721888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.742097 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:54.743297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.805565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1320/2611] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:54.813195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.813549 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.814416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.834707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:54.835937 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.843784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1292/2612] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:54.848640 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.858341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5f48b-9576-4abd-801b-6d245d942a50 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.863637 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.864281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.864568 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.864845 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.865494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:54.865798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.867416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.879374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1321/2613] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:54.883651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.893309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d30f5-d482-44d1-9332-47a811610e00 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.898453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.899069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.899348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.899617 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.900315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.902231 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.930771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1293/2614] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:54 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:54.935304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.945495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216286-7bea-4386-aee5-4ba8daf5afe6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.954515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:55.006382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.006989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.007264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.007534 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.008205 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.010075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.033980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1322/2615] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:54 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:55.038012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.048211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93d2f4-6b32-44eb-b429-f0f7643e23a8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.052763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.053347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.053623 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.053893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.054531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.056160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.065898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1294/2616] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:55.069772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.080765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f9a3f-5e9d-4071-a1ee-0c1ab6573d54 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.085574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9935e5-11e2-439e-82fa-f5bf72ba70e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.086162 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9935e5-11e2-439e-82fa-f5bf72ba70e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.086443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9935e5-11e2-439e-82fa-f5bf72ba70e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.086716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9935e5-11e2-439e-82fa-f5bf72ba70e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.087354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9935e5-11e2-439e-82fa-f5bf72ba70e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.088964 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9935e5-11e2-439e-82fa-f5bf72ba70e3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.100929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1323/2617] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:55.105188 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.115287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd0d4-365c-48e9-88ea-f199459f3a29 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.119860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.120537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.120818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.121100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] PATH_INFO: `/v3/projects/e0a94d66799642029f7c2a71d5d1998b/users/efd009e8d1d64ae292074dafeab503c8/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.135484 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a94d66799642029f7c2a71d5d1998b, user_id=efd009e8d1d64ae292074dafeab503c8, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.138163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.151745 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1295/2618] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:55 2022] PUT /identity/v3/projects/e0a94d66799642029f7c2a71d5d1998b/users/efd009e8d1d64ae292074dafeab503c8/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:55.163674 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.163995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.164298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.191353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] MFA Rules not processed for user `efd009e8d1d64ae292074dafeab503c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:55.192644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.196375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.198477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.218676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:55.264368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1324/2619] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:55.269288 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.281402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2688685-0359-4118-8c8e-a4ebf5d3e3e3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.288585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.289132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.289392 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.289666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.291379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.334462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bdd6b-edfd-451d-b041-9a72f90f6363 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.339277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.339888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.340231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.340537 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.341195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.343093 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.352147 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.353734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.368257 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1296/2620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:44:55.370548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1325/2621] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:44:55.374176 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.386128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0e337-fd94-4c5e-99b9-f91230e062de service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.447145 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.447785 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.448102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.448379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.449015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.450974 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.475889 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1297/2622] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:55.479717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.490596 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e6c5-5d75-4502-a6b8-f963972b91fd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.495350 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.495972 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.496268 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.496859 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.497181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.498814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.500271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:55.512055 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1326/2623] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:55.516375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.527367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4825e-6973-4696-85e2-409e86488c8f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.532976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.534220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.534547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.534821 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] PATH_INFO: `/v3/projects/189b85fbe8f6493cad23f00e0f39b36a/users/61200260dd1c497387ef31417a23213a/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.550659 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189b85fbe8f6493cad23f00e0f39b36a, user_id=61200260dd1c497387ef31417a23213a, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.553364 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.567799 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1298/2624] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:55 2022] PUT /identity/v3/projects/189b85fbe8f6493cad23f00e0f39b36a/users/61200260dd1c497387ef31417a23213a/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:55.572302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.582875 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa0e14-63d2-4485-a5f5-bb13da724322 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.653112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d595077-b0cb-462c-af92-e5879d9fa7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.653112 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d595077-b0cb-462c-af92-e5879d9fa7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.653480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d595077-b0cb-462c-af92-e5879d9fa7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.653824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d595077-b0cb-462c-af92-e5879d9fa7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.654543 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d595077-b0cb-462c-af92-e5879d9fa7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.656258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d595077-b0cb-462c-af92-e5879d9fa7d7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.668966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1327/2625] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:55.673351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.692276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea3a97-b9a8-429f-89d1-b82b38914d00 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.699205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.700276 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.701077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.701566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] PATH_INFO: `/v3/projects/189b85fbe8f6493cad23f00e0f39b36a/users/61200260dd1c497387ef31417a23213a/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.704240 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189b85fbe8f6493cad23f00e0f39b36a, user_id=61200260dd1c497387ef31417a23213a, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.707121 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.719994 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1299/2626] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:55 2022] PUT /identity/v3/projects/189b85fbe8f6493cad23f00e0f39b36a/users/61200260dd1c497387ef31417a23213a/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:55.730368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.730656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.730936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.760202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] MFA Rules not processed for user `61200260dd1c497387ef31417a23213a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:55.761454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.829345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1328/2627] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:55.841123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.842856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.853718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:55.903640 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ee018-2c0b-480c-b8cf-b694d3ac1364 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.907320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.908120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.908393 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.908662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.917464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.918896 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.929804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1300/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:56.007919 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.019267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b0c03-8104-4e12-b90f-e36c751e7054 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.023265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560f37d-c0cc-4412-844b-dfc4b98b98e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.023874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560f37d-c0cc-4412-844b-dfc4b98b98e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.024192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560f37d-c0cc-4412-844b-dfc4b98b98e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.024466 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560f37d-c0cc-4412-844b-dfc4b98b98e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.081913 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560f37d-c0cc-4412-844b-dfc4b98b98e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.083256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560f37d-c0cc-4412-844b-dfc4b98b98e2 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.092244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1329/2629] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:05.010621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.073713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9d7a-7ba1-4af8-af17-25dfd6eca21d service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.078205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.078841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.079114 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.079381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] PATH_INFO: `/v3/users/efd009e8d1d64ae292074dafeab503c8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.081098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd009e8d1d64ae292074dafeab503c8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.082988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.127629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd009e8d1d64ae292074dafeab503c8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.134846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd009e8d1d64ae292074dafeab503c8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.147389 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd009e8d1d64ae292074dafeab503c8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.148044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd009e8d1d64ae292074dafeab503c8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.154919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd009e8d1d64ae292074dafeab503c8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.160918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1301/2630] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:05 2022] DELETE /identity/v3/users/efd009e8d1d64ae292074dafeab503c8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:05.165373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.165829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.165924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.184652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:05.185899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.196008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:05.244608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1330/2631] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:05.255058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.256491 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.288783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:05.315634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4082-cd6e-476a-93e0-5b4c6fd86ba1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.319262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.319865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.320194 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.320466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.329286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.331373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.340751 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1302/2632] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:05.612013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.613577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.626681 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4084d-9320-49d7-bd54-3efe0bb8acee None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.631482 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.632776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.632776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.632776 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] PATH_INFO: `/v3/projects/e0a94d66799642029f7c2a71d5d1998b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.633799 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a94d66799642029f7c2a71d5d1998b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.635722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.683526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a94d66799642029f7c2a71d5d1998b', 'request_id': 'req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.689880 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a94d66799642029f7c2a71d5d1998b', 'request_id': 'req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.697228 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a94d66799642029f7c2a71d5d1998b', 'request_id': 'req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.708567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] The token cache is being invalidate because project e0a94d66799642029f7c2a71d5d1998b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:05.708873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.709496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.711013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1331/2633] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:05 2022] DELETE /identity/v3/projects/e0a94d66799642029f7c2a71d5d1998b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:05.715091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.725321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a56ed8-552e-4bef-b901-fbf3d12d3789 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.780902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.781419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.781692 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.781958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] PATH_INFO: `/v3/users/61200260dd1c497387ef31417a23213a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.783181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61200260dd1c497387ef31417a23213a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.785131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.832328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61200260dd1c497387ef31417a23213a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.839713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61200260dd1c497387ef31417a23213a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.852170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61200260dd1c497387ef31417a23213a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.852791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61200260dd1c497387ef31417a23213a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.859746 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-548effb7-c611-4d7e-862a-b1f3fbbe4628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61200260dd1c497387ef31417a23213a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.866320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1303/2634] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:05 2022] DELETE /identity/v3/users/61200260dd1c497387ef31417a23213a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:06.152517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.154010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.168272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0ccdf-89bd-466a-bcb7-c6ed63f29259 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.227041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.227671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.228005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.228249 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] PATH_INFO: `/v3/projects/189b85fbe8f6493cad23f00e0f39b36a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.229458 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] RBAC: Authorizing `identity:delete_project(project_id=189b85fbe8f6493cad23f00e0f39b36a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.231468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.272300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:06.279092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189b85fbe8f6493cad23f00e0f39b36a', 'request_id': 'req-1aea596e-ec08-4925-8769-409373d6356a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.286736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189b85fbe8f6493cad23f00e0f39b36a', 'request_id': 'req-1aea596e-ec08-4925-8769-409373d6356a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.294174 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189b85fbe8f6493cad23f00e0f39b36a', 'request_id': 'req-1aea596e-ec08-4925-8769-409373d6356a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.304484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] The token cache is being invalidate because project 189b85fbe8f6493cad23f00e0f39b36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:06.304775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.305405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1aea596e-ec08-4925-8769-409373d6356a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.306886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1332/2635] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:06 2022] DELETE /identity/v3/projects/189b85fbe8f6493cad23f00e0f39b36a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:06.311901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.312390 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.312511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.332770 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:06.333965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.378480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:06.395060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1304/2636] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:06.403091 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.403461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.403826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.424945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:06.426164 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.434027 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1333/2637] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:06.438689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.448754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1769b59-2ea9-42bc-9e1c-f32a73b0d475 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.453657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.454251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.454532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.455373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.456125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:06.456463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.458232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.471457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1305/2638] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:06.475866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.485087 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b01ea-b2a8-4d83-81c9-29d4ad7984b6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.489945 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.490542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.490824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.491100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.491722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.493636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.523465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1334/2639] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:06.527628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.537787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e3297-969d-4d76-84aa-5721f1f38a0a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.594602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.595196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.596016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.596500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.597015 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.598936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.620228 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1306/2640] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:06.624289 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.634441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4015-08e4-4902-9b78-e459bc9d39af admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.639062 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.639658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.639965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.640248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.640888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.642588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.652717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1335/2641] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:06.656447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.666868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43bfd-295d-49d5-a625-1861e7f49766 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.671431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee87e7-bd10-4916-85e9-1f0ddceadd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.672054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee87e7-bd10-4916-85e9-1f0ddceadd36 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.672336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee87e7-bd10-4916-85e9-1f0ddceadd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.672608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee87e7-bd10-4916-85e9-1f0ddceadd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.673246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee87e7-bd10-4916-85e9-1f0ddceadd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.675690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee87e7-bd10-4916-85e9-1f0ddceadd36 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.686102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1307/2642] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:06.690299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.700194 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9334bbe-fc57-458d-a742-ec5eeabac025 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.704720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.705564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.705854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.706130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] PATH_INFO: `/v3/projects/82a462fdbc1746bfa17e0806fa2a1a3b/users/accca906008646119ce15e9e67476f6d/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.720193 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a462fdbc1746bfa17e0806fa2a1a3b, user_id=accca906008646119ce15e9e67476f6d, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.723504 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.737083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1336/2643] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:06 2022] PUT /identity/v3/projects/82a462fdbc1746bfa17e0806fa2a1a3b/users/accca906008646119ce15e9e67476f6d/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:06.748607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.748918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.749197 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.777945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] MFA Rules not processed for user `accca906008646119ce15e9e67476f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:06.780175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.841567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1308/2644] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:06.846581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.907514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8bc90-95a6-400f-a6ac-68f29d6520e4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.913630 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.914226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.914505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.914775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.915587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.917723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.943938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1337/2645] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:06.948160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.960060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2d818-871f-4c72-be3c-010c27be5799 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.004477 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:07.018639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.019243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.019518 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.019804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.020445 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.022600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.044613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1309/2646] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:07.047899 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.059422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa77cc6-8653-447f-b23d-3f92dc672377 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.068759 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.069405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.069687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.069966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.070619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.072256 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.083412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1338/2647] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:07.087757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.098123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe85468-e138-4d9b-9229-ee5127ae72ff admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.102822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.103483 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.103763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.104075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] PATH_INFO: `/v3/projects/182148d74ef74afaac53f497d5c7b9e7/users/20d29bd836d047108142b4636ff54d22/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.120784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182148d74ef74afaac53f497d5c7b9e7, user_id=20d29bd836d047108142b4636ff54d22, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.123563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.137873 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1310/2648] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:07 2022] PUT /identity/v3/projects/182148d74ef74afaac53f497d5c7b9e7/users/20d29bd836d047108142b4636ff54d22/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:07.142014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.152347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c8319-ee2c-4c60-b41c-474d175d8d98 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.192627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:07.214696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce0f5-6780-4726-9906-855500042655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.215303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce0f5-6780-4726-9906-855500042655 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.215588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce0f5-6780-4726-9906-855500042655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.216006 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce0f5-6780-4726-9906-855500042655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.216654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ce0f5-6780-4726-9906-855500042655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.218236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ce0f5-6780-4726-9906-855500042655 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.230590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1339/2649] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:07.234500 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.245865 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b091b7-d4a4-4397-9c3a-e6739c6f98c4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.250809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.251462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.252464 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.252739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] PATH_INFO: `/v3/projects/182148d74ef74afaac53f497d5c7b9e7/users/20d29bd836d047108142b4636ff54d22/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.254646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=182148d74ef74afaac53f497d5c7b9e7, user_id=20d29bd836d047108142b4636ff54d22, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.257313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.268592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1311/2650] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:07 2022] PUT /identity/v3/projects/182148d74ef74afaac53f497d5c7b9e7/users/20d29bd836d047108142b4636ff54d22/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:07.278873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.279161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.279440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.308336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] MFA Rules not processed for user `20d29bd836d047108142b4636ff54d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:07.309547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.377540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1340/2651] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:07.388876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.390595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.457048 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6649eb-ca96-4c84-b5dd-26aab3a04adf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.460666 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.461266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.461540 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.461811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.470806 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.472246 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.481512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1312/2652] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:07.557873 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.568383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739c2db-d735-4282-b699-4b8b0625dacb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.571965 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205750-6ce9-46c8-ab72-8b0c0500ac14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.572556 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205750-6ce9-46c8-ab72-8b0c0500ac14 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.572832 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205750-6ce9-46c8-ab72-8b0c0500ac14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.573101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205750-6ce9-46c8-ab72-8b0c0500ac14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.628353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205750-6ce9-46c8-ab72-8b0c0500ac14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.629706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205750-6ce9-46c8-ab72-8b0c0500ac14 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.638298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1341/2653] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:10.354969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.416968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271074f-56f3-4ed5-9aa4-ccd5c610fc59 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.421443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.422060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.422338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.422605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] PATH_INFO: `/v3/users/accca906008646119ce15e9e67476f6d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.423748 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=accca906008646119ce15e9e67476f6d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.425681 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.470708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:10.473929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'accca906008646119ce15e9e67476f6d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.481510 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'accca906008646119ce15e9e67476f6d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.493353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'accca906008646119ce15e9e67476f6d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.493982 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'accca906008646119ce15e9e67476f6d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.501297 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'accca906008646119ce15e9e67476f6d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.507771 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1313/2654] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:10 2022] DELETE /identity/v3/users/accca906008646119ce15e9e67476f6d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:10.512421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.512909 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.513014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.534641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:10.535860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.710694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:10.719488 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1342/2655] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:10.729535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.730944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.792259 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10db39-b614-4e2a-b559-27f4f161e975 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.795998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.796645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.796929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.797204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.806708 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.808166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.817795 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1314/2656] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:11.090199 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.091625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:11.103979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc425a6-6762-473a-ae1c-feafafe73336 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.108443 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.109067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.109339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.109612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] PATH_INFO: `/v3/projects/82a462fdbc1746bfa17e0806fa2a1a3b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.110769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a462fdbc1746bfa17e0806fa2a1a3b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.113359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.160052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a462fdbc1746bfa17e0806fa2a1a3b', 'request_id': 'req-a338eda1-3b30-4930-9cc6-58bca815991e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.166137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a462fdbc1746bfa17e0806fa2a1a3b', 'request_id': 'req-a338eda1-3b30-4930-9cc6-58bca815991e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.172920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a462fdbc1746bfa17e0806fa2a1a3b', 'request_id': 'req-a338eda1-3b30-4930-9cc6-58bca815991e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.184138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] The token cache is being invalidate because project 82a462fdbc1746bfa17e0806fa2a1a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:11.184434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.185056 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.186509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1343/2657] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:11 2022] DELETE /identity/v3/projects/82a462fdbc1746bfa17e0806fa2a1a3b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:11.190704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.200262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fc88-8416-4cbf-b251-db394dc86bfa service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.258597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.259224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.259501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.259772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] PATH_INFO: `/v3/users/20d29bd836d047108142b4636ff54d22` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.260912 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d29bd836d047108142b4636ff54d22)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.262843 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.309663 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d29bd836d047108142b4636ff54d22'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.317084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d29bd836d047108142b4636ff54d22'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.324920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:11.330918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d29bd836d047108142b4636ff54d22'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.331753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d29bd836d047108142b4636ff54d22'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.338813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f2c706f4-6100-439b-ba64-f545f2a81187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d29bd836d047108142b4636ff54d22'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.344660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1315/2658] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:11 2022] DELETE /identity/v3/users/20d29bd836d047108142b4636ff54d22 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:11.627854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.629680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:11.643250 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338eda1-3b30-4930-9cc6-58bca815991e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.701122 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.701676 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.701957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.702232 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] PATH_INFO: `/v3/projects/182148d74ef74afaac53f497d5c7b9e7` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.703331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=182148d74ef74afaac53f497d5c7b9e7)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.705309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.749806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182148d74ef74afaac53f497d5c7b9e7', 'request_id': 'req-136fefde-832e-405d-8809-7299530e82cd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.756399 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182148d74ef74afaac53f497d5c7b9e7', 'request_id': 'req-136fefde-832e-405d-8809-7299530e82cd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.763851 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182148d74ef74afaac53f497d5c7b9e7', 'request_id': 'req-136fefde-832e-405d-8809-7299530e82cd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.774389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] The token cache is being invalidate because project 182148d74ef74afaac53f497d5c7b9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:11.774680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.775299 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-136fefde-832e-405d-8809-7299530e82cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:11.776783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1344/2659] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:11 2022] DELETE /identity/v3/projects/182148d74ef74afaac53f497d5c7b9e7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:11.782347 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.783379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.783476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.803277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:11.804512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:11.865331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1316/2660] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:11.874083 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.874368 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.874700 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.893997 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:11.895201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:11.903498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1345/2661] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:11.908337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.918784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a8b8a-3144-4114-8ec6-bed2c25aebf2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.923970 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.924780 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.925057 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.925329 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.926005 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:11.926309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.928089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.940725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1317/2662] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:11.945044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.954353 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79db37-e827-42e1-97fa-14ec63c925b2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.959203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.959807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.960113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.960383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.961021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.962885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.967664 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:11.995680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1346/2663] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:11 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:11.999869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.010146 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeca369-1979-416a-a34a-3206255d0cf0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.066812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.067409 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.067683 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.067968 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.068598 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.070492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.091305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1318/2664] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:11 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:12.095925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.106402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2ac52-2293-4349-9e37-26e0db0feb87 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.111742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.112522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.112824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.113118 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.113811 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.115394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.125440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1347/2665] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:12.129641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.139593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ab51c-2894-4389-8da6-e601cce8ff13 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.144774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b4dc9-2330-480a-946f-0b0e3f3c1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.145367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4dc9-2330-480a-946f-0b0e3f3c1c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.145664 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4dc9-2330-480a-946f-0b0e3f3c1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.145916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b4dc9-2330-480a-946f-0b0e3f3c1c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.146809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b4dc9-2330-480a-946f-0b0e3f3c1c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.148536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b4dc9-2330-480a-946f-0b0e3f3c1c5f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.159717 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1319/2666] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:12.163660 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.173485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275659f0-17e2-4039-be91-05e0ae35e50f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.177998 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.178639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.178917 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.179192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] PATH_INFO: `/v3/projects/bf406828c9fa4694a786e803d22e47e7/users/4166c6375c614b3ebb48674fc0455277/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.193416 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf406828c9fa4694a786e803d22e47e7, user_id=4166c6375c614b3ebb48674fc0455277, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.196066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.209254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1348/2667] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:12 2022] PUT /identity/v3/projects/bf406828c9fa4694a786e803d22e47e7/users/4166c6375c614b3ebb48674fc0455277/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:12.219479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.219875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.220251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.247359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] MFA Rules not processed for user `4166c6375c614b3ebb48674fc0455277`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:12.248682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:12.288914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:12.316408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1320/2668] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:12.320961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.380900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529c7a-9126-40a2-a4b1-71452212be17 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.385496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.386098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.386379 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.386653 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.387276 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.389185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.416695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1349/2669] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:12 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:12.420782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.432131 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0ebb6-5101-4891-8a57-2bc7308929e5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.488335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.488867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.489149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.489423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.490044 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.491963 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.513266 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1321/2670] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:12 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:12.517610 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.528926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440c090-57b8-4aff-a1ca-2f47fa2c7333 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.534417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.535009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.535290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.535565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.536235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.537849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.547549 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1350/2671] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:12.552285 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.562293 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f824ef8-99e4-40f0-8fbd-1a42880d4a2a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.567021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.567677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.567989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.568275 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] PATH_INFO: `/v3/projects/fa8f7e3bb41a4afba20cae703a606719/users/f153ea5d691048f890233fb53bcff881/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.583727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8f7e3bb41a4afba20cae703a606719, user_id=f153ea5d691048f890233fb53bcff881, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.586541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.601579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1322/2672] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:12 2022] PUT /identity/v3/projects/fa8f7e3bb41a4afba20cae703a606719/users/f153ea5d691048f890233fb53bcff881/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:12.605498 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.616064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0807b-f39d-4658-9a82-3a87fc45dd70 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.671242 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e04da8-6600-4272-8cc3-ad3c4bbea7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.671938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04da8-6600-4272-8cc3-ad3c4bbea7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.672132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04da8-6600-4272-8cc3-ad3c4bbea7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.672405 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e04da8-6600-4272-8cc3-ad3c4bbea7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.673032 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e04da8-6600-4272-8cc3-ad3c4bbea7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.675338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e04da8-6600-4272-8cc3-ad3c4bbea7d4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.685766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1351/2673] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:12.689978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.700583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4690b-653d-4f23-8cf9-1002f0f8c84c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.705279 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.705920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.706202 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.706480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] PATH_INFO: `/v3/projects/fa8f7e3bb41a4afba20cae703a606719/users/f153ea5d691048f890233fb53bcff881/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.708467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8f7e3bb41a4afba20cae703a606719, user_id=f153ea5d691048f890233fb53bcff881, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.711086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.722020 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1323/2674] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:12 2022] PUT /identity/v3/projects/fa8f7e3bb41a4afba20cae703a606719/users/f153ea5d691048f890233fb53bcff881/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:12.731920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.732237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.732515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.761001 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] MFA Rules not processed for user `f153ea5d691048f890233fb53bcff881`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:12.762145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:12.770328 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:12.829854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1352/2675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:12.840424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:12.842166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:12.899803 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a485f-88c4-4d43-ae6d-6f135963482b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:12.903438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ca02d-77b0-4af1-bfae-0d811c3ba1d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:12.904063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ca02d-77b0-4af1-bfae-0d811c3ba1d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:12.904343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ca02d-77b0-4af1-bfae-0d811c3ba1d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:12.904615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ca02d-77b0-4af1-bfae-0d811c3ba1d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:12.914535 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ca02d-77b0-4af1-bfae-0d811c3ba1d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:12.915992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ca02d-77b0-4af1-bfae-0d811c3ba1d5 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:12.927330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1324/2676] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:14.320734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:14.386341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab53ae-8d33-451a-b9ba-e961c29ed893 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:14.390876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:14.391481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:14.391763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:14.392639 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] PATH_INFO: `/v3/users/4166c6375c614b3ebb48674fc0455277` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:14.393847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4166c6375c614b3ebb48674fc0455277)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:14.395739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:14.442175 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4166c6375c614b3ebb48674fc0455277'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:14.448898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4166c6375c614b3ebb48674fc0455277'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:14.449561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4166c6375c614b3ebb48674fc0455277'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:14.456377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4166c6375c614b3ebb48674fc0455277'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:14.467817 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4166c6375c614b3ebb48674fc0455277'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:14.474047 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1353/2677] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:14 2022] DELETE /identity/v3/users/4166c6375c614b3ebb48674fc0455277 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:14.478586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:14.479088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:14.479186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:14.498360 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:14.499566 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:14.530603 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:14.563165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1325/2678] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:14.573627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:14.575013 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:14.602008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:14.633290 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa12bf-60a6-4cbd-a7de-24d78004bd29 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:14.636878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:14.637480 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:14.637827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:14.638131 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:14.647314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:14.648877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:14.658304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1354/2679] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:14.922610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:14.925226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:14.937550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b89349-9fd2-4ef8-a9c3-dd5464fca612 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:14.942522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:14.943150 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:14.943430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:14.943734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] PATH_INFO: `/v3/projects/bf406828c9fa4694a786e803d22e47e7` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:14.944899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf406828c9fa4694a786e803d22e47e7)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:14.946807 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:14.995669 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf406828c9fa4694a786e803d22e47e7', 'request_id': 'req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.002486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf406828c9fa4694a786e803d22e47e7', 'request_id': 'req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.008119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf406828c9fa4694a786e803d22e47e7', 'request_id': 'req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.020662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] The token cache is being invalidate because project bf406828c9fa4694a786e803d22e47e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:15.020978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.021881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.023478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1326/2680] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:14 2022] DELETE /identity/v3/projects/bf406828c9fa4694a786e803d22e47e7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:15.027628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.038303 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895bff3-794c-432f-8af8-bef0e8fa11bf service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.092514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.093135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.093414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.093694 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] PATH_INFO: `/v3/users/f153ea5d691048f890233fb53bcff881` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.094795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f153ea5d691048f890233fb53bcff881)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.097492 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:15.144069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f153ea5d691048f890233fb53bcff881'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.151553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f153ea5d691048f890233fb53bcff881'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.152336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f153ea5d691048f890233fb53bcff881'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.160580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f153ea5d691048f890233fb53bcff881'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.175075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-074568b2-a737-4f02-8cca-6712cca8c3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f153ea5d691048f890233fb53bcff881'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.181559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1355/2681] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:15 2022] DELETE /identity/v3/users/f153ea5d691048f890233fb53bcff881 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:15.474471 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.476433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:15.503526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2da55c-2ffc-4b1d-826f-5bb228c6f336 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.555816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba003e-0138-4985-b17c-653adacfe9aa None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.556342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba003e-0138-4985-b17c-653adacfe9aa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.556454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba003e-0138-4985-b17c-653adacfe9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.576697 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.577337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.577618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.577897 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] PATH_INFO: `/v3/projects/fa8f7e3bb41a4afba20cae703a606719` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.578584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-74ba003e-0138-4985-b17c-653adacfe9aa None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:15.579027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa8f7e3bb41a4afba20cae703a606719)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.581060 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-74ba003e-0138-4985-b17c-653adacfe9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:15.581900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:15.630722 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:15.637985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa8f7e3bb41a4afba20cae703a606719', 'request_id': 'req-6775a46c-1d27-444f-8204-c75db4b100bf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.645090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa8f7e3bb41a4afba20cae703a606719', 'request_id': 'req-6775a46c-1d27-444f-8204-c75db4b100bf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.645499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1356/2682] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:45:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:15.649378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:15.651938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa8f7e3bb41a4afba20cae703a606719', 'request_id': 'req-6775a46c-1d27-444f-8204-c75db4b100bf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.667206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] The token cache is being invalidate because project fa8f7e3bb41a4afba20cae703a606719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:15.667569 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.668273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.669996 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1327/2683] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:15 2022] DELETE /identity/v3/projects/fa8f7e3bb41a4afba20cae703a606719 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:15.675849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8df2f-cbe2-4cb4-a3b6-eb88d16a02a0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.676203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8df2f-cbe2-4cb4-a3b6-eb88d16a02a0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.676499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8df2f-cbe2-4cb4-a3b6-eb88d16a02a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.696762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.697322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f6d8df2f-cbe2-4cb4-a3b6-eb88d16a02a0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:15.698237 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:15.698496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f6d8df2f-cbe2-4cb4-a3b6-eb88d16a02a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:15.759446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1357/2684] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:15.768528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.768760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a46c-1d27-444f-8204-c75db4b100bf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.768893 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.769084 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.774031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.774755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.775035 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.775304 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.777204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:15.789287 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:15.790521 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:15.798999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1358/2685] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:15.803104 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.812030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a28f-52b5-47af-b376-5cebb02302bb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.817562 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.818575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.818871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.819150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.819798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:15.820161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.821752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:15.833605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1359/2686] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:45:15.837788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.838349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.839995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:15.847831 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f70dd-547e-4600-9c75-f6f70433d500 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.848987 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1328/2687] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:45:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:45:15.852844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.853345 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.853742 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.853915 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.854514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.856430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:15.893835 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1360/2688] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:15 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:15.900544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.919386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182dee4-ea79-4fc0-9911-981021d3d4d4 service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.995023 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.995638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.995922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.996226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.996874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.998888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.024977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1329/2689] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:15 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:16.029720 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:16.045204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b7030-aaa4-4ecf-bde9-400c4bd23204 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:16.050749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:16.051386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.051691 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.053185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.053185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:16.054557 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.066978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1361/2690] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:16.071016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:16.082186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7407ff0-44dd-47a4-80ab-5b685e736ba6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:16.087009 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5103d6-d3b7-4087-b6dc-2c1df3269204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:16.087613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5103d6-d3b7-4087-b6dc-2c1df3269204 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.087895 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5103d6-d3b7-4087-b6dc-2c1df3269204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.088204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5103d6-d3b7-4087-b6dc-2c1df3269204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.088876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5103d6-d3b7-4087-b6dc-2c1df3269204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:16.090505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5103d6-d3b7-4087-b6dc-2c1df3269204 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.106639 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1330/2691] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:16.112586 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:16.123731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8476a-357a-4280-8893-05bc8259ef46 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:16.132348 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:16.133022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.133305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.133582 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] PATH_INFO: `/v3/projects/c60d18a2d0d04572acfca594078d8542/users/0aa9aad1649e446b9fcf3a81a376b7e7/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.153240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60d18a2d0d04572acfca594078d8542, user_id=0aa9aad1649e446b9fcf3a81a376b7e7, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:16.155898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.174703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1362/2692] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:16 2022] PUT /identity/v3/projects/c60d18a2d0d04572acfca594078d8542/users/0aa9aad1649e446b9fcf3a81a376b7e7/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:16.185981 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.187779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.188300 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.203481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:16.228503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] MFA Rules not processed for user `0aa9aad1649e446b9fcf3a81a376b7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:16.229904 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:16.338229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1331/2693] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:16.351530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:16.353511 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:16.442601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23407c4-84e5-4a94-907d-91bbb1c6ecdd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:16.447155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:16.448021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.448494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.448955 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.461914 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:16.463535 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.476685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1363/2694] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:18.447517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:18.449098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:18.559449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc6efd-9dc6-4550-8462-35f6bbeed2af None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:18.567893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:18.568831 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:18.569245 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:18.569653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:18.572196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:18.590590 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:18.595791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:18.615193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1332/2695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:25.433265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:25.434672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:25.457241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:25.496130 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254be849-2ed7-4ebe-92ac-13be60751946 service glance] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:25.499778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:25.500414 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:25.500695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:25.501559 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:25.503336 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:25.512979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:25.514387 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4062c6c-f2b0-466b-a090-45bb3c9ee0bb service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:25.523818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1364/2696] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:28.602881 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=63 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:30.078769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:30.081090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:30.152774 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63725e4-406c-4225-97dc-200cd0331b62 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:30.158523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:30.159128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:30.159410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:30.159689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:30.161479 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:30.173019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:30.174616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:30.184609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1333/2697] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:31.781003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:31.781457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:31.781547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:31.800505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:31.801684 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:31.855583 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1365/2698] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:45:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:31.867481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:31.934447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5cf3-d0ae-4e4a-9bc6-86bb78677436 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:31.939561 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876864a-43dc-444a-b468-46744f77dbda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:31.940205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876864a-43dc-444a-b468-46744f77dbda service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:31.940485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876864a-43dc-444a-b468-46744f77dbda service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:31.940761 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876864a-43dc-444a-b468-46744f77dbda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:31.949065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876864a-43dc-444a-b468-46744f77dbda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:31.950646 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876864a-43dc-444a-b468-46744f77dbda service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:31.959596 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1334/2699] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:45:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:45:33.478342 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:33.554828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aab46-6317-4661-b876-ff1de7e26661 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:33.559592 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:33.560275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.560689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.560981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] PATH_INFO: `/v3/users/0aa9aad1649e446b9fcf3a81a376b7e7` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.562203 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa9aad1649e446b9fcf3a81a376b7e7)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:33.564377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:33.621195 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa9aad1649e446b9fcf3a81a376b7e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.630657 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa9aad1649e446b9fcf3a81a376b7e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.631383 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa9aad1649e446b9fcf3a81a376b7e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.640437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa9aad1649e446b9fcf3a81a376b7e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.647143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:33.657985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa9aad1649e446b9fcf3a81a376b7e7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.664686 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1366/2700] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:33 2022] DELETE /identity/v3/users/0aa9aad1649e446b9fcf3a81a376b7e7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:33.669374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58360-513b-4981-832a-516b3a39e95f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.669856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58360-513b-4981-832a-516b3a39e95f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.669978 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58360-513b-4981-832a-516b3a39e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.690755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-71b58360-513b-4981-832a-516b3a39e95f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:33.692084 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-71b58360-513b-4981-832a-516b3a39e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:33.812481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1335/2701] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:33.823536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:33.825985 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:33.935226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4f5aa-1f07-44d6-b696-b409b8577bb4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:33.940725 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557da73-b0fd-4fba-88a8-fdb8cbdc4b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:33.941340 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557da73-b0fd-4fba-88a8-fdb8cbdc4b36 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.941616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557da73-b0fd-4fba-88a8-fdb8cbdc4b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.941889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557da73-b0fd-4fba-88a8-fdb8cbdc4b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.959649 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557da73-b0fd-4fba-88a8-fdb8cbdc4b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:33.961163 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557da73-b0fd-4fba-88a8-fdb8cbdc4b36 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:33.976046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1367/2702] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:34.324787 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58360-513b-4981-832a-516b3a39e95f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.326239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-71b58360-513b-4981-832a-516b3a39e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:34.341867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58360-513b-4981-832a-516b3a39e95f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.347052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.347701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.348016 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.348295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] PATH_INFO: `/v3/projects/c60d18a2d0d04572acfca594078d8542` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.350181 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60d18a2d0d04572acfca594078d8542)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.352327 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.370249 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:34.412707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60d18a2d0d04572acfca594078d8542', 'request_id': 'req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:34.425525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60d18a2d0d04572acfca594078d8542', 'request_id': 'req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:34.433946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60d18a2d0d04572acfca594078d8542', 'request_id': 'req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:34.458862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] The token cache is being invalidate because project c60d18a2d0d04572acfca594078d8542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:34.459503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:34.462499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ec3144dc-9efe-4d3f-a453-1561d96ef6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:34.464631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1336/2703] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:34 2022] DELETE /identity/v3/projects/c60d18a2d0d04572acfca594078d8542 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:34.470916 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.471204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.471514 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.504717 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:34.507023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:34.568430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1368/2704] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:34.577255 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.577602 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.577945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.598861 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:34.600091 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:34.607979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1337/2705] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:34.612200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.622216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa97da7-a4c4-4014-bc45-47217460fb3f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.627839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.628518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.628828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.629103 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.629825 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:34.630408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.632053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.650475 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1369/2706] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:34.651648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.661880 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae61195-0c17-4cf6-84d8-bf7008229f66 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.666863 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.667492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.668086 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.668552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.669478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.672322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.705339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1338/2707] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:34.709812 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.720396 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabf341-e1ec-4b9f-977c-ffb83c8dfb2d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.793494 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.794109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.794389 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.794663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.795292 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.798685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.832364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1370/2708] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:34.836956 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.857155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2331f1-5663-4ff9-bf57-fd1db42bf557 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.863422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.864100 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.864422 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.864711 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.865423 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.867109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.881628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1339/2709] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:34.886415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.896135 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:34.903778 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10b18-dc0d-469a-b771-a47f235e7f83 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.910329 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dcd05-77ca-41de-bf16-6a75dd4d6c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.911173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dcd05-77ca-41de-bf16-6a75dd4d6c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.913565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dcd05-77ca-41de-bf16-6a75dd4d6c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.913896 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dcd05-77ca-41de-bf16-6a75dd4d6c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.914723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dcd05-77ca-41de-bf16-6a75dd4d6c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.917064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dcd05-77ca-41de-bf16-6a75dd4d6c76 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.934254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1371/2710] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:34.939366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.949900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389b990-f75f-4660-8dda-a46ee2de923f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.973918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.975134 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.994902 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.996054 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] PATH_INFO: `/v3/projects/00a45ba3e79b425493703f960803cea1/users/623e7860c4ee4918b7a917024d35e39b/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.037791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a45ba3e79b425493703f960803cea1, user_id=623e7860c4ee4918b7a917024d35e39b, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.040601 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.069353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1340/2711] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:34 2022] PUT /identity/v3/projects/00a45ba3e79b425493703f960803cea1/users/623e7860c4ee4918b7a917024d35e39b/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:35.071161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d38c6-9002-46cb-8506-07d2c178fd24 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.071547 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d38c6-9002-46cb-8506-07d2c178fd24 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.071963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d38c6-9002-46cb-8506-07d2c178fd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.104515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1b5d38c6-9002-46cb-8506-07d2c178fd24 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:35.109057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1b5d38c6-9002-46cb-8506-07d2c178fd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.171866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1372/2712] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:35.183758 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.185627 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.244773 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddf8ff-ac9d-4f5e-839c-54424c672412 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.248473 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.249073 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.249354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.249630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.258665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.260111 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.269609 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1341/2713] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:41.724656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:41.725306 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:41.726015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:41.761025 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] MFA Rules not processed for user `623e7860c4ee4918b7a917024d35e39b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:41.762435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:41.834109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1373/2714] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:41.838605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:41.849854 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4bbaf-1740-4474-84bf-f3a738771462 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:41.855835 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:41.856546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:41.856817 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:41.857122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:41.857769 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:41.859764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:41.888976 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1342/2715] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:41 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:41.893907 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:41.904824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d87bb6-4f3c-405e-a7d6-35fcc57c6355 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:41.959332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:41.959999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:41.960243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:41.960513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:41.961127 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:41.963007 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:41.983677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1374/2716] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:41 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:41.988475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:41.999013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904654f-0d40-4e37-a43d-75c3c7f373c6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.003707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.004379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.004685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.004951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.005600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.007229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.018574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1343/2717] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:42.023254 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.034783 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce99fa-5a44-4866-996d-efdb5e37a4a8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.039040 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.039698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.040012 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.040291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] PATH_INFO: `/v3/projects/df54202140b64b32b80027a51c770571/users/c2d4267b2f3e4b908a14d7f70d6452b5/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.052386 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:42.056821 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df54202140b64b32b80027a51c770571, user_id=c2d4267b2f3e4b908a14d7f70d6452b5, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.059545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.074731 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1375/2718] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:42 2022] PUT /identity/v3/projects/df54202140b64b32b80027a51c770571/users/c2d4267b2f3e4b908a14d7f70d6452b5/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:42.079154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.089529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184933f1-7310-46b8-a995-18345e1e66d7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.094766 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:42.149260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.150029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.150330 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.150607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.151251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.152877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.162721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1344/2719] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:42.166878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.177974 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29f209-8772-4d13-8414-5ebf173d8826 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.182541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfc2a5-3511-465c-b136-ca52d70c187a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.183180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfc2a5-3511-465c-b136-ca52d70c187a admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.183461 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfc2a5-3511-465c-b136-ca52d70c187a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.183739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfc2a5-3511-465c-b136-ca52d70c187a admin admin] PATH_INFO: `/v3/projects/df54202140b64b32b80027a51c770571/users/c2d4267b2f3e4b908a14d7f70d6452b5/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.185769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfc2a5-3511-465c-b136-ca52d70c187a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df54202140b64b32b80027a51c770571, user_id=c2d4267b2f3e4b908a14d7f70d6452b5, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.189079 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfc2a5-3511-465c-b136-ca52d70c187a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.200005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1376/2720] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:42 2022] PUT /identity/v3/projects/df54202140b64b32b80027a51c770571/users/c2d4267b2f3e4b908a14d7f70d6452b5/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:46.762156 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:46.764287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:46.961634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086da52-ff1e-4f78-9e6a-f198285f32d9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:47.149691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:47.151984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:47.153485 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:47.153876 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:47.156782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:47.173874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:47.175585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f9d2a-42b7-43d3-a9ed-2961b258ba2a service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:47.210431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1345/2721] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:47.615402 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d867aa-5ad5-4008-bfa7-3718f3e74908 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:47.616403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d867aa-5ad5-4008-bfa7-3718f3e74908 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:47.616839 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d867aa-5ad5-4008-bfa7-3718f3e74908 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:47.618354 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1377/2722] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:45:47.628368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ed3b4-8267-46a7-86b1-bb62c458282e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:47.628809 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ed3b4-8267-46a7-86b1-bb62c458282e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:47.629225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ed3b4-8267-46a7-86b1-bb62c458282e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:47.664113 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0d3ed3b4-8267-46a7-86b1-bb62c458282e None None] MFA Rules not processed for user `505ea6d7aac346e1a604b192a2a0c364`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:47.667206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0d3ed3b4-8267-46a7-86b1-bb62c458282e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:47.722524 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:47.761747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1346/2723] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:47.789677 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d867aa-5ad5-4008-bfa7-3718f3e74908 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:47.791579 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b9d867aa-5ad5-4008-bfa7-3718f3e74908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:47.823526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d867aa-5ad5-4008-bfa7-3718f3e74908 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:47.841857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929af1-691a-4622-9bbd-1dc4df19866e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:47.842513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929af1-691a-4622-9bbd-1dc4df19866e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:47.842819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929af1-691a-4622-9bbd-1dc4df19866e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:47.843098 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929af1-691a-4622-9bbd-1dc4df19866e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:47.863364 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd929af1-691a-4622-9bbd-1dc4df19866e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:47.864894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd929af1-691a-4622-9bbd-1dc4df19866e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:47.901304 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1378/2724] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:48.084296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093eb8d-4a40-4e79-851c-92f115bd3831 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:48.085586 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093eb8d-4a40-4e79-851c-92f115bd3831 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:48.086031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093eb8d-4a40-4e79-851c-92f115bd3831 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:48.087634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1347/2725] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 16:45:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:45:48.100199 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f49d6-5e32-4b14-abff-5d1574cdec6a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:48.100561 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f49d6-5e32-4b14-abff-5d1574cdec6a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:48.100867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f49d6-5e32-4b14-abff-5d1574cdec6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:48.156283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5b7f49d6-5e32-4b14-abff-5d1574cdec6a None None] MFA Rules not processed for user `f1ab09f1996244a2b328a887fe91febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:48.164259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5b7f49d6-5e32-4b14-abff-5d1574cdec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:48.179110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1379/2726] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 16:45:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:48.200607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093eb8d-4a40-4e79-851c-92f115bd3831 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:48.230201 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093eb8d-4a40-4e79-851c-92f115bd3831 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:48.240251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6bedc-0394-45a4-b3bf-87a05770574e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:48.240697 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6bedc-0394-45a4-b3bf-87a05770574e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:48.241100 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6bedc-0394-45a4-b3bf-87a05770574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:48.242340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:48.243164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:48.243608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:48.244076 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:48.267775 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:48.269295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:48.274531 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ddb6bedc-0394-45a4-b3bf-87a05770574e None None] MFA Rules not processed for user `bb5d3834f04142e288c04cc05665329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:48.276036 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ddb6bedc-0394-45a4-b3bf-87a05770574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:48.290165 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1348/2727] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:48.358652 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1380/2728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:48.823742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:48.852882 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dbe8a-1093-4c3b-86ee-7ced45bbf2e6 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:48.859314 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:48.861541 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:48.861879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:48.862162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:48.864321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:48.878527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:48.881537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:48.893501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1349/2729] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:49.161248 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3a9e-f91b-4aad-a5d5-aee9298c9cfb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:49.161738 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3a9e-f91b-4aad-a5d5-aee9298c9cfb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:49.161855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3a9e-f91b-4aad-a5d5-aee9298c9cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:49.201281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-32ca3a9e-f91b-4aad-a5d5-aee9298c9cfb None None] MFA Rules not processed for user `c2d4267b2f3e4b908a14d7f70d6452b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:49.205322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-32ca3a9e-f91b-4aad-a5d5-aee9298c9cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:49.299999 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1381/2730] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:45:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:49.310544 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:49.312209 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:49.381729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b197ed-cea5-4ae9-91f4-17a7f5195d9c service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:49.386303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3671de0f76cc475db615dcc9b351c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:49.387092 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:49.387475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:49.387856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:49.496492 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:49.498384 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:49.523783 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1350/2731] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:49.783341 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca3a0c-10be-4e41-b519-72a01cca0117 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:49.783693 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca3a0c-10be-4e41-b519-72a01cca0117 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:49.783995 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca3a0c-10be-4e41-b519-72a01cca0117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:49.809585 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bdca3a0c-10be-4e41-b519-72a01cca0117 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:49.810840 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bdca3a0c-10be-4e41-b519-72a01cca0117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:49.898807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:49.921223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1382/2732] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:45:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:49.935239 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:49.936764 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:50.065116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadcba9-85a4-4e45-a002-8ff2389d2c3d service glance] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:50.071729 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:50.072410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:50.072688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:50.072992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:50.084723 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:50.088802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:50.091410 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:50.103027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1351/2733] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:45:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:45:53.517344 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a457c9-953f-43f6-915d-f6d11e0a68d1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:53.517922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a457c9-953f-43f6-915d-f6d11e0a68d1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:53.518108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a457c9-953f-43f6-915d-f6d11e0a68d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:53.569209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c9a457c9-953f-43f6-915d-f6d11e0a68d1 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:53.570468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9a457c9-953f-43f6-915d-f6d11e0a68d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:53.607191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1383/2734] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:53.650660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:53.676520 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5b32-5ee3-49f1-95c5-fb0e55835224 service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:53.729438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3add05e-1502-4ab5-8a15-c9e2771c02e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:53.735354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add05e-1502-4ab5-8a15-c9e2771c02e5 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:53.735653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add05e-1502-4ab5-8a15-c9e2771c02e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:53.737436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add05e-1502-4ab5-8a15-c9e2771c02e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:53.782846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3add05e-1502-4ab5-8a15-c9e2771c02e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:53.794591 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3add05e-1502-4ab5-8a15-c9e2771c02e5 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:53.816102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1352/2735] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:46:01.005685 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:01.006301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:01.006301 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:01.027291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] MFA Rules not processed for user `623e7860c4ee4918b7a917024d35e39b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:01.034074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:01.041231 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1384/2736] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:03.074043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:03.074580 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:03.074814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:03.095403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:03.096969 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:03.105143 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1353/2737] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:03.119272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:03.134505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292fef7-704a-4586-b5c3-d63bd064c06f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:03.139723 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:03.141216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:03.141566 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:03.141911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:03.151704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:03.153648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:03.165411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1385/2738] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:46:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:46:13.089789 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=11 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:13.160515 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=16 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:31.902808 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:31.913012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531a2db-44af-4353-80b2-aea14dcf83b8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:31.916727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:31.917349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:31.917607 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:31.917877 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:31.927909 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:31.929431 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:31.939583 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1354/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:37.444767 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:37.454806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f24e-c7ed-478d-b849-ea81a2761384 service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:37.458418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:37.459005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:37.459283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:37.459558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:37.461347 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:37.525003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:37.526430 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:37.537576 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1386/2740] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:40.782689 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:40.784262 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:40.800233 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b11e-41fc-48ba-9729-a089e6921a63 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:40.804737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:40.805359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:40.805638 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:40.805919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] PATH_INFO: `/v3/users/79d593b826c64087bac576d2204302b4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:40.807034 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79d593b826c64087bac576d2204302b4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:40.808977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:40.857215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79d593b826c64087bac576d2204302b4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:40.865328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79d593b826c64087bac576d2204302b4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:40.877595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79d593b826c64087bac576d2204302b4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:40.878662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79d593b826c64087bac576d2204302b4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:40.886716 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1378e8a9-5580-4d54-ae71-68b8d77290b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79d593b826c64087bac576d2204302b4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:40.894636 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1355/2741] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:46:40 2022] DELETE /identity/v3/users/79d593b826c64087bac576d2204302b4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:41.079136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.090587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe61c1c-4758-42b3-92fa-15ddce356b25 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.147667 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.148265 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.148546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.148823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] PATH_INFO: `/v3/projects/cdf93612acce4a868bf2105a6567855b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.149933 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf93612acce4a868bf2105a6567855b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.151844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.200349 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf93612acce4a868bf2105a6567855b', 'request_id': 'req-63811f10-1e74-48d1-ae24-089cea15f814'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:41.206981 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf93612acce4a868bf2105a6567855b', 'request_id': 'req-63811f10-1e74-48d1-ae24-089cea15f814'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:41.213892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf93612acce4a868bf2105a6567855b', 'request_id': 'req-63811f10-1e74-48d1-ae24-089cea15f814'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:41.224670 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] The token cache is being invalidate because project cdf93612acce4a868bf2105a6567855b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:46:41.224953 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:41.225563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63811f10-1e74-48d1-ae24-089cea15f814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:41.227048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1387/2742] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:46:41 2022] DELETE /identity/v3/projects/cdf93612acce4a868bf2105a6567855b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:46:41.231985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.232273 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.232546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.252144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:41.253332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:41.311496 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1356/2743] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:41.320798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.321095 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.321373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.341233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:41.342457 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:41.350427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1388/2744] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:41.354624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.365078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e188-174f-463d-9a31-f94dddc6882d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.369742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.370342 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.370645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.370918 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.371567 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:46:41.371869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.373499 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.385752 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1357/2745] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:46:41.389901 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.405703 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11524c-a50a-49cb-ab19-9099fdd3288d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.410615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.411217 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.411499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.411772 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.412450 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.414370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.430574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:41.446629 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1389/2746] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:41 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:46:41.450814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.460853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00664048-6150-4a7e-bcc9-945befdcb2b0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.505662 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:41.520992 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.521650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.521935 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.522212 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.522848 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.524805 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.549179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1358/2747] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:41 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:41.553882 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.564644 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbefeb5-98bd-4d4e-9696-9c85375b0411 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.569227 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.569807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.570082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.570373 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.571005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.572624 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.582360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1390/2748] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:41.586682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.597331 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447130d8-773c-4893-b3b2-b610dc6350e3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.601870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d940589-cc23-4395-9a6b-7df480a72d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.602447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d940589-cc23-4395-9a6b-7df480a72d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.602727 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d940589-cc23-4395-9a6b-7df480a72d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.603003 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d940589-cc23-4395-9a6b-7df480a72d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.603647 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d940589-cc23-4395-9a6b-7df480a72d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.605320 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d940589-cc23-4395-9a6b-7df480a72d7a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.617045 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1359/2749] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:41.621145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.632045 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c495029-fcac-405b-8920-295c30ae145b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.636599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.637236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.637518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.637796 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] PATH_INFO: `/v3/projects/ebd03bbf32154ccabb8d233f00347081/users/481ae20977e8424cb460d8108905577d/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.652562 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd03bbf32154ccabb8d233f00347081, user_id=481ae20977e8424cb460d8108905577d, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.655171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.668072 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1391/2750] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:41 2022] PUT /identity/v3/projects/ebd03bbf32154ccabb8d233f00347081/users/481ae20977e8424cb460d8108905577d/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:41.673842 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d28ebf-18ae-41c1-94e9-47d347ef127d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.674124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d28ebf-18ae-41c1-94e9-47d347ef127d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.674394 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d28ebf-18ae-41c1-94e9-47d347ef127d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.693557 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b5d28ebf-18ae-41c1-94e9-47d347ef127d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:41.694728 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b5d28ebf-18ae-41c1-94e9-47d347ef127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:41.753339 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1360/2751] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:41.763052 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:41.764797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:41.821068 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb30ae-c042-4991-805f-b33baa190b31 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:41.824695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:41.825305 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:41.825584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:41.825858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:41.835444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:41.836898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:41.846219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1392/2752] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:45.846461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1299e8e-6ebd-49e6-afbb-11f4a390ba1a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:45.846946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1299e8e-6ebd-49e6-afbb-11f4a390ba1a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:45.847047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1299e8e-6ebd-49e6-afbb-11f4a390ba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:45.881268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f1299e8e-6ebd-49e6-afbb-11f4a390ba1a None None] MFA Rules not processed for user `481ae20977e8424cb460d8108905577d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:45.882608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f1299e8e-6ebd-49e6-afbb-11f4a390ba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:45.956106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1361/2753] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:45.971710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:45.973189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:46.052467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6e2d3-cb63-451b-8e80-dc267b006cc6 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:46.058043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:46.058658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:46.058939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:46.059219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:46.068886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:46.070499 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:46.080733 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1393/2754] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:47.368760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:47.369274 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:47.369414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:47.388378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] MFA Rules not processed for user `481ae20977e8424cb460d8108905577d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:47.389599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:47.397505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1362/2755] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:56.075971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:57.382757 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:30.233417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:30.234857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:30.373449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d889b53-cb43-4013-9d59-140a71ee5a44 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:30.379225 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:30.379824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:30.380136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:30.380412 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:30.382143 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:30.394202 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:30.395760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:30.405540 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1394/2756] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:34.138680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:34.161850 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c17084-decb-4c4b-b1ab-e64a477d4ce2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:34.165832 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:34.166523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:34.166802 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:34.167077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:34.168869 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:34.227587 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:34.229060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:34.238829 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1363/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:39.039233 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:39.050377 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eeae9-c896-4115-b574-809001f6315d service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:39.054226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:39.054852 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:39.055185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:39.055437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:39.057307 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:39.095616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:39.097542 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:39.116408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1395/2758] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:44.234088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=34 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:45.441161 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:45.443905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:45.477204 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2fc29-b3f8-41bf-adc9-7588aa83f3cd service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:45.482017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:45.482654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:45.482934 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:45.483210 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] PATH_INFO: `/v3/users/9fb81fa5765c44a0afa8d9fb4786272c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:45.484411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb81fa5765c44a0afa8d9fb4786272c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:45.486354 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:45.533615 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb81fa5765c44a0afa8d9fb4786272c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.541495 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb81fa5765c44a0afa8d9fb4786272c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.553730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb81fa5765c44a0afa8d9fb4786272c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.554359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb81fa5765c44a0afa8d9fb4786272c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.561298 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb81fa5765c44a0afa8d9fb4786272c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.567703 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1364/2759] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:47:45 2022] DELETE /identity/v3/users/9fb81fa5765c44a0afa8d9fb4786272c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:45.747855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:45.759522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b79a6-07f2-4e2b-ac3a-d08b00d8ef50 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:45.801014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:45.822246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:45.822911 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:45.823192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:45.823473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] PATH_INFO: `/v3/projects/cd41122fd4e94aa786798d8f54214f0a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:45.824618 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd41122fd4e94aa786798d8f54214f0a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:45.826505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:45.872533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd41122fd4e94aa786798d8f54214f0a', 'request_id': 'req-f7a5124f-47b3-4cc7-aae8-bba72e535546'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.879490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd41122fd4e94aa786798d8f54214f0a', 'request_id': 'req-f7a5124f-47b3-4cc7-aae8-bba72e535546'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.886562 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd41122fd4e94aa786798d8f54214f0a', 'request_id': 'req-f7a5124f-47b3-4cc7-aae8-bba72e535546'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.897190 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] The token cache is being invalidate because project cd41122fd4e94aa786798d8f54214f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:47:45.897478 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.898109 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:45.899625 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1396/2760] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:47:45 2022] DELETE /identity/v3/projects/cd41122fd4e94aa786798d8f54214f0a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:45.903657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:45.905162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:45.918075 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350e923-e185-47c6-8891-7cb179e312b3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:45.974512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:45.975177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:45.975456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:45.975732 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] PATH_INFO: `/v3/users/5dd839a82aae4ce4a94fee87a85a1b34` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:45.976941 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd839a82aae4ce4a94fee87a85a1b34)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:45.978864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:46.032428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd839a82aae4ce4a94fee87a85a1b34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.040425 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd839a82aae4ce4a94fee87a85a1b34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.053536 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd839a82aae4ce4a94fee87a85a1b34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.054218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd839a82aae4ce4a94fee87a85a1b34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.056448 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:46.062260 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-55329621-9f90-4229-bf9d-c0fddc68e9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd839a82aae4ce4a94fee87a85a1b34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.070838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1365/2761] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:47:45 2022] DELETE /identity/v3/users/5dd839a82aae4ce4a94fee87a85a1b34 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:47:46.261009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:46.263049 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:46.293807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5124f-47b3-4cc7-aae8-bba72e535546 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:46.353971 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:46.354606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:46.354888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:46.355165 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] PATH_INFO: `/v3/projects/01e3335bc64744e1b2ee2fe5c061f881` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:46.356325 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e3335bc64744e1b2ee2fe5c061f881)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:46.358216 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:46.409066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e3335bc64744e1b2ee2fe5c061f881', 'request_id': 'req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.416476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e3335bc64744e1b2ee2fe5c061f881', 'request_id': 'req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.428375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e3335bc64744e1b2ee2fe5c061f881', 'request_id': 'req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.436391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:46.440768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] The token cache is being invalidate because project 01e3335bc64744e1b2ee2fe5c061f881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:47:46.441518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.442277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:46.444003 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1397/2762] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:47:46 2022] DELETE /identity/v3/projects/01e3335bc64744e1b2ee2fe5c061f881 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:55.651682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.652193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.652193 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.682055 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] MFA Rules not processed for user `cd94a4b5dfbf4af6984f8bcf4fb8593b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:55.683251 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:55.746571 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1366/2763] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:55.787298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:55.788897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:55.858022 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e65e8-e3e1-41ed-b14f-67a7ac9d614e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:55.863538 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1e90301e20d4d4ea7f11c7bef1ef4c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:55.864161 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:55.864440 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:55.864711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:55.872491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:55.874044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:55.882192 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1398/2764] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 16:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:47:57.688793 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:57.691159 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:57.804250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c998-4e5e-4437-9508-b65bd71faa2c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:57.808676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:57.809490 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:57.809921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:57.810343 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:57.812478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:57.889656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:57.891369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:57.904241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1367/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:03.045235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.046992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:03.115337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f0d70-c5f7-4902-bc36-3a83ab7f574d service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.119752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.120467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.120751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.121028 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] PATH_INFO: `/v3/users/623e7860c4ee4918b7a917024d35e39b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.122160 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=623e7860c4ee4918b7a917024d35e39b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.124108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.170439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '623e7860c4ee4918b7a917024d35e39b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.177757 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '623e7860c4ee4918b7a917024d35e39b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.178417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '623e7860c4ee4918b7a917024d35e39b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.185647 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '623e7860c4ee4918b7a917024d35e39b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.198277 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '623e7860c4ee4918b7a917024d35e39b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.204158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1399/2766] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:03 2022] DELETE /identity/v3/users/623e7860c4ee4918b7a917024d35e39b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:03.379814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.391229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109ff97-b04f-4c37-80be-0f39359f7944 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.405211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:03.451922 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.452574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.452852 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.453123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] PATH_INFO: `/v3/projects/00a45ba3e79b425493703f960803cea1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.454230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a45ba3e79b425493703f960803cea1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.457116 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.504424 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a45ba3e79b425493703f960803cea1', 'request_id': 'req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.510430 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a45ba3e79b425493703f960803cea1', 'request_id': 'req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.515933 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a45ba3e79b425493703f960803cea1', 'request_id': 'req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.528337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] The token cache is being invalidate because project 00a45ba3e79b425493703f960803cea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:03.528628 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.529248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.530722 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1368/2767] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:03 2022] DELETE /identity/v3/projects/00a45ba3e79b425493703f960803cea1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:03.535298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.536760 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:03.551597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60f8de-178f-4324-a3f6-67acf167fe18 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.556251 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:03.610483 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.611138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.611967 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.612444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] PATH_INFO: `/v3/users/c2d4267b2f3e4b908a14d7f70d6452b5` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.613577 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d4267b2f3e4b908a14d7f70d6452b5)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.615501 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:03.660636 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d4267b2f3e4b908a14d7f70d6452b5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.667565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d4267b2f3e4b908a14d7f70d6452b5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.668318 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d4267b2f3e4b908a14d7f70d6452b5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.676876 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d4267b2f3e4b908a14d7f70d6452b5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.688724 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-532f7110-0791-403f-b7e9-ea52a0e79cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d4267b2f3e4b908a14d7f70d6452b5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:03.694950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1400/2768] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:48:03 2022] DELETE /identity/v3/users/c2d4267b2f3e4b908a14d7f70d6452b5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:48:03.874006 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:03.875702 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:03.903232 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bb1b5-37b9-420f-a54f-665dbcffc8f8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:03.965268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:03.965891 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:03.966166 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:03.966437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] PATH_INFO: `/v3/projects/df54202140b64b32b80027a51c770571` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:03.967534 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=df54202140b64b32b80027a51c770571)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:03.969454 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:04.014503 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:04.015753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df54202140b64b32b80027a51c770571', 'request_id': 'req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:04.024516 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df54202140b64b32b80027a51c770571', 'request_id': 'req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:04.030376 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df54202140b64b32b80027a51c770571', 'request_id': 'req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:04.043175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] The token cache is being invalidate because project df54202140b64b32b80027a51c770571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:48:04.043461 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:04.044119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:48:04.045649 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1369/2769] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:48:03 2022] DELETE /identity/v3/projects/df54202140b64b32b80027a51c770571 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:48:05.027771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037ce26-6898-47dc-a792-27321b9922a5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:05.028356 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037ce26-6898-47dc-a792-27321b9922a5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:05.028356 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037ce26-6898-47dc-a792-27321b9922a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:05.048509 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f037ce26-6898-47dc-a792-27321b9922a5 None None] MFA Rules not processed for user `bb5d3834f04142e288c04cc05665329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:05.050581 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f037ce26-6898-47dc-a792-27321b9922a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:05.103609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1401/2770] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:48:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:14.040059 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:15.097220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:24.067319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:24.068837 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:24.139767 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8fe9-fbff-4e40-925c-b0a288a8cbcf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:24.143353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:24.143986 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:24.144271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:24.145050 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:24.147019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:24.230186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:24.231610 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:24.251777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1370/2771] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:28.515609 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.516173 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.516298 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.534956 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] MFA Rules not processed for user `481ae20977e8424cb460d8108905577d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:28.536530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:28.592807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1402/2772] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:49:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:28.902288 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:28.911624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cc4d0-87da-471f-9a31-659b9fed739a service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:28.915634 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:28.916290 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:28.916568 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:28.916834 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:28.918641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:28.929154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:28.930582 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:28.939078 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1371/2773] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:29.276874 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:29.287890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6f29-4174-42f7-8d62-f0fd1c04ef99 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:29.292200 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:29.292714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:29.292993 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:29.293270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:29.295411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:29.309900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:29.312101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:29.328129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1403/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:38.935267 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=74 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:39.318879 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=31 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:05.321466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:05.322998 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:05.340694 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad37f7-fce3-4535-b737-a51754281059 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:05.346337 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:05.346940 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:05.347220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:05.347494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:05.350252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:05.363093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:05.365351 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:05.376308 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1372/2775] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:08.680642 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:08.693281 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146500a3-e74e-4f07-bf50-49ded8a171df service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:08.697599 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:08.698141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:08.698420 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:08.698710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:08.700495 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:08.763824 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:08.765258 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:08.775222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1404/2776] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:11.874324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:11.875762 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:11.951411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f3f3c-c3fc-49bb-b22c-31b18a464dfa service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:11.956701 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:11.957303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:11.957579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:11.957849 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:11.959838 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:11.971517 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:11.973069 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:11.986213 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1373/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:15.049856 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.063367 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacd338-c3c0-4de9-a9bb-6edb88996749 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.069057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.070070 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.070361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.070721 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.072435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:15.091580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.093316 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.114088 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.117526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:15.120313 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1405/2778] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:15.135593 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27443a3-ea74-49b3-b811-8fa37c654222 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.150099 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.151220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.151220 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.151690 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] PATH_INFO: `/v3/users/bb5d3834f04142e288c04cc05665329f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.153065 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5d3834f04142e288c04cc05665329f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.155594 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.213486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5d3834f04142e288c04cc05665329f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.221531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5d3834f04142e288c04cc05665329f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.235333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5d3834f04142e288c04cc05665329f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.236094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5d3834f04142e288c04cc05665329f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.243369 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5d3834f04142e288c04cc05665329f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.249225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1374/2779] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:15 2022] DELETE /identity/v3/users/bb5d3834f04142e288c04cc05665329f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:15.424627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.435528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a02806-0180-478d-a220-bba10744dd4d service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.492623 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.493282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.493565 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.493865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] PATH_INFO: `/v3/projects/7c99de6ca9094fa7bd119a3be6817725` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.494992 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c99de6ca9094fa7bd119a3be6817725)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.496927 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.543888 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c99de6ca9094fa7bd119a3be6817725', 'request_id': 'req-94db2e11-e952-4b87-8709-38967202e0f4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.550775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c99de6ca9094fa7bd119a3be6817725', 'request_id': 'req-94db2e11-e952-4b87-8709-38967202e0f4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.558442 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c99de6ca9094fa7bd119a3be6817725', 'request_id': 'req-94db2e11-e952-4b87-8709-38967202e0f4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.569424 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] The token cache is being invalidate because project 7c99de6ca9094fa7bd119a3be6817725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:15.569706 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.570332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.571798 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1406/2780] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:15 2022] DELETE /identity/v3/projects/7c99de6ca9094fa7bd119a3be6817725 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:15.575715 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.577177 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:15.590862 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc08b2-55be-441c-8a5d-14746a2e6e6c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.635284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:15.649090 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.649713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.650012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.650282 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] PATH_INFO: `/v3/users/f7aad9b95e804d4a93ae6975ee52b39b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.651415 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7aad9b95e804d4a93ae6975ee52b39b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.653348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.698403 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7aad9b95e804d4a93ae6975ee52b39b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.705579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7aad9b95e804d4a93ae6975ee52b39b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.718616 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7aad9b95e804d4a93ae6975ee52b39b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.719283 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7aad9b95e804d4a93ae6975ee52b39b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.726957 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7aad9b95e804d4a93ae6975ee52b39b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:15.734984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1375/2781] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:15 2022] DELETE /identity/v3/users/f7aad9b95e804d4a93ae6975ee52b39b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:15.909106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.910741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:15.924324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2e11-e952-4b87-8709-38967202e0f4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.933696 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:15.983889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.984550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.984833 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.985111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] PATH_INFO: `/v3/projects/2bf4dc9d2b8140ba8de7cf0b15d5fd63` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.986246 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf4dc9d2b8140ba8de7cf0b15d5fd63)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.988220 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:16.044734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf4dc9d2b8140ba8de7cf0b15d5fd63', 'request_id': 'req-b4581069-4301-4481-ac32-34572e53357a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:16.051234 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf4dc9d2b8140ba8de7cf0b15d5fd63', 'request_id': 'req-b4581069-4301-4481-ac32-34572e53357a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:16.058692 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf4dc9d2b8140ba8de7cf0b15d5fd63', 'request_id': 'req-b4581069-4301-4481-ac32-34572e53357a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:16.068837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] The token cache is being invalidate because project 2bf4dc9d2b8140ba8de7cf0b15d5fd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:16.069116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:16.069728 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:16.071191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1407/2782] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:15 2022] DELETE /identity/v3/projects/2bf4dc9d2b8140ba8de7cf0b15d5fd63 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:25.730243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:26.065781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:48.690284 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:48.691938 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:48.708707 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e514fcf-71b9-4c95-abf1-d177e6bff81b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:48.765537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:48.766163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:48.766447 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:48.766725 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] PATH_INFO: `/v3/users/481ae20977e8424cb460d8108905577d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:48.767845 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=481ae20977e8424cb460d8108905577d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:48.769890 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:48.817305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481ae20977e8424cb460d8108905577d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:48.825126 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481ae20977e8424cb460d8108905577d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:48.837542 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481ae20977e8424cb460d8108905577d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:48.838175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481ae20977e8424cb460d8108905577d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:48.844914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-889607bb-79f5-46ee-80b1-8d913b6308ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481ae20977e8424cb460d8108905577d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:48.850945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1376/2783] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:48 2022] DELETE /identity/v3/users/481ae20977e8424cb460d8108905577d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:49.044891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.047302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.061140 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581069-4301-4481-ac32-34572e53357a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.119627 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.120286 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.120570 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.120848 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] PATH_INFO: `/v3/projects/ebd03bbf32154ccabb8d233f00347081` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.121976 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd03bbf32154ccabb8d233f00347081)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.123877 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.170092 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd03bbf32154ccabb8d233f00347081', 'request_id': 'req-52c1ca00-5bc8-4da8-8f20-17317703f225'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:49.176078 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd03bbf32154ccabb8d233f00347081', 'request_id': 'req-52c1ca00-5bc8-4da8-8f20-17317703f225'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:49.183513 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd03bbf32154ccabb8d233f00347081', 'request_id': 'req-52c1ca00-5bc8-4da8-8f20-17317703f225'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:49.193780 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] The token cache is being invalidate because project ebd03bbf32154ccabb8d233f00347081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:49.194066 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:49.194687 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-52c1ca00-5bc8-4da8-8f20-17317703f225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:49.196615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1408/2784] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:49 2022] DELETE /identity/v3/projects/ebd03bbf32154ccabb8d233f00347081 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:49.201529 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.201813 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.202083 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.390402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:49.391612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.436185 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:49.453030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1377/2785] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:49.460826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.461189 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.461471 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.481157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:49.482395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.490362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1409/2786] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:49.495441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.504972 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bd1c2-0a7c-4366-9b5d-e01c6e1a61f9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.510053 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.510641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.510916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.511546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.512243 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:49.512546 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.514381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.526142 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1378/2787] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:50:49.530300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.539837 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6beb-0e6a-4b03-ae26-8dfbc8f11f83 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.544451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.545038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.545317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.545588 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.546240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.548141 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.584274 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1410/2788] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:49 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:49.588381 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.598332 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5b0eb-3760-46f1-8a0c-4855d3cb9f80 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.656367 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.656887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.657171 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.657444 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.658068 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.659983 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.681563 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1379/2789] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:49 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:49.686105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.697213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5e443-9297-454d-9ae5-37acb208a609 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.702042 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.702633 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.702908 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.703207 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.703925 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.705580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.716094 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1411/2790] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:49.719839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.730532 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6667bd-cf0a-4f5a-b02f-ed4e92342d73 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.735032 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dd8f1-e754-4047-944c-8ce61ce7971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.735622 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dd8f1-e754-4047-944c-8ce61ce7971a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.735899 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dd8f1-e754-4047-944c-8ce61ce7971a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.736205 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dd8f1-e754-4047-944c-8ce61ce7971a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.736898 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dd8f1-e754-4047-944c-8ce61ce7971a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.738608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dd8f1-e754-4047-944c-8ce61ce7971a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.748719 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1380/2791] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:49.752867 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.762751 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e123af-88f9-45b1-a801-c75a75dc2359 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.767781 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.768469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.768747 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.769019 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] PATH_INFO: `/v3/projects/eec14f7fdaae43b68bc7113a7f17544b/users/0de20b3d10414699bec5a6eaf15f1aa3/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.783847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec14f7fdaae43b68bc7113a7f17544b, user_id=0de20b3d10414699bec5a6eaf15f1aa3, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.786476 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.799795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1412/2792] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:49 2022] PUT /identity/v3/projects/eec14f7fdaae43b68bc7113a7f17544b/users/0de20b3d10414699bec5a6eaf15f1aa3/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:49.804739 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e00b-adc5-4fc1-adab-f0f8bc7d5476 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.805051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e00b-adc5-4fc1-adab-f0f8bc7d5476 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.805335 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44e00b-adc5-4fc1-adab-f0f8bc7d5476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.823747 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1c44e00b-adc5-4fc1-adab-f0f8bc7d5476 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:49.824985 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1c44e00b-adc5-4fc1-adab-f0f8bc7d5476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.882439 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1381/2793] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:49.892525 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:49.894309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:49.915484 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:49.952517 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c48f2-b492-45dd-ace6-53786c2014ef admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:49.956275 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:49.956887 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:49.957169 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:49.957447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:49.967553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:49.968989 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:49.978803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1413/2794] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:53.884366 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a3eb4-887f-4843-b168-159f935d1427 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:53.884905 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a3eb4-887f-4843-b168-159f935d1427 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:53.885007 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a3eb4-887f-4843-b168-159f935d1427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:53.913348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-091a3eb4-887f-4843-b168-159f935d1427 None None] MFA Rules not processed for user `0de20b3d10414699bec5a6eaf15f1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:53.915853 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-091a3eb4-887f-4843-b168-159f935d1427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:53.962488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:53.984348 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1382/2795] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:50:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:53.999370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:54.001422 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:54.084803 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c295bad-48fc-484f-b9dc-4284eda3113d service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:54.090795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:54.091434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:54.091858 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:54.092176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:54.100648 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:54.103272 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:54.113085 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1414/2796] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:55.500676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:55.501170 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:55.501271 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:55.520860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] MFA Rules not processed for user `0de20b3d10414699bec5a6eaf15f1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:55.522058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:55.529432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1383/2797] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:50:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:57.173157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:57.176046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:57.281658 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633aa51-50f0-4bea-8f97-dc7bf6ebff7b service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:57.287755 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:57.288578 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:57.288983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:57.289449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:57.291990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:57.304459 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:57.307283 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:57.335360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1415/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:02.392109 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:02.402359 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907907fe-d501-4cd5-8e9c-b3bbd3cce647 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:02.405921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:02.406522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:02.406804 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:02.407082 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:02.409818 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:02.420269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:02.421812 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:02.431914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1384/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:07.325453 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:08.179854 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:08.201197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac70141-75e5-4d8c-ae48-39f682fc1946 service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:08.206631 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:08.207226 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:08.207505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:08.207779 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:08.209569 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:08.221906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:08.223474 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a04c7-6f8e-4eb1-807b-eea3113fac90 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:08.234411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1416/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:12.370875 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:12.372531 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:12.388945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd4856-057c-498c-9937-5d507652e94c service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:12.394486 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ab09f1996244a2b328a887fe91febd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:12.395158 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:12.395528 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:12.395872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:12.397678 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:12.411106 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:12.412827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:12.425693 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1385/2801] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:18.229969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:22.420574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=22 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:58.611197 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e0121-20c3-459e-b0f1-ed7ebd1bc651 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.611763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e0121-20c3-459e-b0f1-ed7ebd1bc651 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.611763 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e0121-20c3-459e-b0f1-ed7ebd1bc651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.630970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fe0e0121-20c3-459e-b0f1-ed7ebd1bc651 None None] MFA Rules not processed for user `0de20b3d10414699bec5a6eaf15f1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:58.632224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fe0e0121-20c3-459e-b0f1-ed7ebd1bc651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:58.640069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1417/2802] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:51:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:08.625695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:36.223093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:36.224527 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:36.238505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fa534-6b23-42a2-be65-618c29b8ad9f service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:36.243438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:36.244064 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:36.244349 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:36.245230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:36.246984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:36.258305 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:36.259866 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd03a1-db31-4fed-a97a-e2b23e5dca35 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:36.270599 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1386/2803] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:46.266481 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:57.162016 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:57.162590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:57.162590 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:57.184365 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] MFA Rules not processed for user `0de20b3d10414699bec5a6eaf15f1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:57.185564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:57.192437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1418/2804] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:07.179053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:00.371115 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:00.371673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:00.371673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:00.390860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] MFA Rules not processed for user `0de20b3d10414699bec5a6eaf15f1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:00.392211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:00.398878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1387/2805] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:54:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:00.707021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:00.708552 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:00.721545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6295a-5b39-460a-805e-ffb9c5edf07e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:00.727204 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:00.727829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:00.728138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:00.728411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:00.730114 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:00.740675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:00.742153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:00.750983 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1419/2806] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:10.385588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:10.746508 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:50.271420 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:50.272859 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:50.328833 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe38531-ddf5-4770-945f-22196c48c5ec None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:50.333307 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:50.333916 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:50.334195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:50.334466 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:50.336252 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:50.349573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:50.350995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:50.361900 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1388/2807] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:57.210943 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:57.212359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:57.224553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea0b9e-e7ad-42b5-b8eb-90d60f452e61 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:57.228176 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:57.228797 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:57.229077 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:57.229462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:57.231179 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:57.245548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:57.246952 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:57.257938 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1420/2808] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:00.357008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=26 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:07.253209 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:09.494713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:09.508437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d52928-6236-497d-bcd3-a74e914f9926 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:09.514606 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:09.515319 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:09.515597 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:09.515871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] PATH_INFO: `/v3/users/0de20b3d10414699bec5a6eaf15f1aa3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:09.518501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de20b3d10414699bec5a6eaf15f1aa3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:09.520478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:09.569966 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de20b3d10414699bec5a6eaf15f1aa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.578164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de20b3d10414699bec5a6eaf15f1aa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.594236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de20b3d10414699bec5a6eaf15f1aa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.596418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de20b3d10414699bec5a6eaf15f1aa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.603819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-128d2ea2-7155-4012-91a9-feabaaaeee94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de20b3d10414699bec5a6eaf15f1aa3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.609822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1389/2809] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:09 2022] DELETE /identity/v3/users/0de20b3d10414699bec5a6eaf15f1aa3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:09.793591 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:09.806054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270689d3-cc4b-48f3-9e7a-97ad0243c867 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:09.867789 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:09.868454 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:09.868736 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:09.869014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] PATH_INFO: `/v3/projects/eec14f7fdaae43b68bc7113a7f17544b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:09.870146 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec14f7fdaae43b68bc7113a7f17544b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:09.876388 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:09.925601 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec14f7fdaae43b68bc7113a7f17544b', 'request_id': 'req-5fd2b223-d705-4fe6-b247-38aec726f256'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.931661 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec14f7fdaae43b68bc7113a7f17544b', 'request_id': 'req-5fd2b223-d705-4fe6-b247-38aec726f256'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.939836 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec14f7fdaae43b68bc7113a7f17544b', 'request_id': 'req-5fd2b223-d705-4fe6-b247-38aec726f256'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.950137 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] The token cache is being invalidate because project eec14f7fdaae43b68bc7113a7f17544b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:09.950426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.951064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5fd2b223-d705-4fe6-b247-38aec726f256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:09.952619 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1421/2810] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:09 2022] DELETE /identity/v3/projects/eec14f7fdaae43b68bc7113a7f17544b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:09.958868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:09.959167 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:09.959449 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:09.978737 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:09.979962 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:10.040946 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1390/2811] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:10.048980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.049263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.049539 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.068427 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:10.069617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:10.078101 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1422/2812] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:10.084226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.105755 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac517d-62be-47ea-a4b3-ecd3ffc02f04 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.110674 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.111373 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.111650 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.111919 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.112592 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:55:10.112893 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.114467 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.125815 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1391/2813] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:55:10.130155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.140528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43508f78-ae94-4fff-b1e2-c6c238c622c4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.146038 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.146643 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.146924 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.147201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.147890 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.149829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.177370 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1423/2814] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:10 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:10.181562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.191714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6b9ba-bb22-4c57-991e-5abed1239a3f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.246691 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.247286 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.247573 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.247847 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.248494 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.250391 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.258119 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:10.273008 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1392/2815] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:10 2022] POST /identity/v3/users => generated 280 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:10.277082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.287889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db0a5-8b64-4889-a51d-b3545a7a83f0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.293000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.293606 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.294029 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.294326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.294977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.296634 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.306866 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1424/2816] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:10.310731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.321175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd809c-cbd9-4262-9e74-3669ebf9bce9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.325742 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19456baa-b10b-46c8-a5fa-27b46b7f6474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.326333 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19456baa-b10b-46c8-a5fa-27b46b7f6474 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.326613 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19456baa-b10b-46c8-a5fa-27b46b7f6474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.326887 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19456baa-b10b-46c8-a5fa-27b46b7f6474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.327525 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19456baa-b10b-46c8-a5fa-27b46b7f6474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.330125 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19456baa-b10b-46c8-a5fa-27b46b7f6474 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.340432 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1393/2817] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:10.344580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.355418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050624-a427-48be-a4b5-07be2735f5cd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.359939 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.361106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.361384 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.361659 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] PATH_INFO: `/v3/projects/1843d874d39a4b6daf27ae2810611004/users/54553a52690b45709d00d871958d2ba3/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.376718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1843d874d39a4b6daf27ae2810611004, user_id=54553a52690b45709d00d871958d2ba3, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.379469 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.394132 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1425/2818] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:10 2022] PUT /identity/v3/projects/1843d874d39a4b6daf27ae2810611004/users/54553a52690b45709d00d871958d2ba3/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:10.458685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.458965 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.459241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.487313 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] MFA Rules not processed for user `54553a52690b45709d00d871958d2ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:10.488709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:10.553093 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1394/2819] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:10.587563 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.589267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:10.620033 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:10.647857 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316be3-38b8-4305-9d15-ecde0412657e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.651390 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9052ca-3bd2-4a33-bcf0-73bba6d05cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.652015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9052ca-3bd2-4a33-bcf0-73bba6d05cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.652297 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9052ca-3bd2-4a33-bcf0-73bba6d05cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.652571 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9052ca-3bd2-4a33-bcf0-73bba6d05cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.661449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9052ca-3bd2-4a33-bcf0-73bba6d05cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.662860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9052ca-3bd2-4a33-bcf0-73bba6d05cf8 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.671403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1426/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:10.687254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.747860 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0485-2b1e-4c86-83c0-1431a9f7f011 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.752442 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.753077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.753350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.753618 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] PATH_INFO: `/v3/users/54553a52690b45709d00d871958d2ba3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.754721 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=54553a52690b45709d00d871958d2ba3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:10.756648 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:10.802281 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54553a52690b45709d00d871958d2ba3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:10.809110 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54553a52690b45709d00d871958d2ba3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:10.820515 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:10.821735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54553a52690b45709d00d871958d2ba3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:10.822537 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54553a52690b45709d00d871958d2ba3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:10.832149 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54553a52690b45709d00d871958d2ba3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:10.837825 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1395/2821] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:10 2022] DELETE /identity/v3/users/54553a52690b45709d00d871958d2ba3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:10.842264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.842734 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.842855 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:10.862158 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:10.863359 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:10.921936 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1427/2822] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:10.932264 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:10.933665 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:10.991462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad42ea-4421-4d97-98bf-d4895ce28f4f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:10.995179 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b051df-dd65-47ad-8202-7e4d7a8f95c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:10.995778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b051df-dd65-47ad-8202-7e4d7a8f95c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:10.996103 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b051df-dd65-47ad-8202-7e4d7a8f95c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:10.996377 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b051df-dd65-47ad-8202-7e4d7a8f95c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.006370 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b051df-dd65-47ad-8202-7e4d7a8f95c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.007811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b051df-dd65-47ad-8202-7e4d7a8f95c0 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.017749 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1396/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:11.288977 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.290462 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:11.303222 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765dcfe-ebfd-4328-b69e-ff6ab70bb81f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.308201 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.309404 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.309871 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.310156 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] PATH_INFO: `/v3/projects/1843d874d39a4b6daf27ae2810611004` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.311417 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1843d874d39a4b6daf27ae2810611004)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.314039 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.363615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1843d874d39a4b6daf27ae2810611004', 'request_id': 'req-efa3c1b8-53c8-46ca-a58b-802cc95dd643'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:11.369666 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1843d874d39a4b6daf27ae2810611004', 'request_id': 'req-efa3c1b8-53c8-46ca-a58b-802cc95dd643'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:11.377168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1843d874d39a4b6daf27ae2810611004', 'request_id': 'req-efa3c1b8-53c8-46ca-a58b-802cc95dd643'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:11.387338 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] The token cache is being invalidate because project 1843d874d39a4b6daf27ae2810611004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:11.387620 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:11.388282 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-efa3c1b8-53c8-46ca-a58b-802cc95dd643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:11.390053 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1428/2824] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:11 2022] DELETE /identity/v3/projects/1843d874d39a4b6daf27ae2810611004 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:11.395914 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.396238 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.396519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.415428 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:11.416653 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:11.478973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1397/2825] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:55:11.487024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.487302 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.487584 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.507641 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:11.508881 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:11.518171 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1429/2826] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:11.522979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.533321 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec674bf8-4d89-42ff-b0a8-54be128342d0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.538051 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.538656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.538929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.539198 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.539840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:55:11.540183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.541810 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.553215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1398/2827] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:55:11.557374 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.566680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87a709-e5d5-4c6e-9212-93a53c9e52c2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.571481 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.572123 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.572398 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.572671 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.573295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.575187 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.602711 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1430/2828] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:11 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:11.606874 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.617081 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981ed16-fd4d-4c0f-865c-ee5b438c1f4a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.673630 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.674224 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.674505 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.674776 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.675402 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.677323 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.698060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1399/2829] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:11 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:55:11.702351 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.713567 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf689d61-c15d-47a8-b42f-87e039b409a9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.718181 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.718775 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.719058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.719332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.720689 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.722295 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.732361 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1431/2830] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:11.736365 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.746130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03d732-165a-40ad-971f-b7e3f0a801f9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.750868 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3359097-535f-46b7-8c33-e4b639136a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.751612 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359097-535f-46b7-8c33-e4b639136a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.751888 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359097-535f-46b7-8c33-e4b639136a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.752192 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359097-535f-46b7-8c33-e4b639136a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.752858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3359097-535f-46b7-8c33-e4b639136a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.754506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3359097-535f-46b7-8c33-e4b639136a8b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.755754 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:11.767010 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1400/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:11.770950 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.780746 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11baf0d4-79b5-4f1a-9ef0-fb43595cd8bf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.786126 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.786769 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.787155 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.787441 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] PATH_INFO: `/v3/projects/96c05ee821094a619097e92d0ff108c5/users/76fc5d21a28a49b3896367cd2cd7dc49/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.798818 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:11.803269 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c05ee821094a619097e92d0ff108c5, user_id=76fc5d21a28a49b3896367cd2cd7dc49, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:11.806806 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:11.819898 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1432/2832] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:11 2022] PUT /identity/v3/projects/96c05ee821094a619097e92d0ff108c5/users/76fc5d21a28a49b3896367cd2cd7dc49/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:11.829885 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.830164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.830434 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:11.857324 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] MFA Rules not processed for user `76fc5d21a28a49b3896367cd2cd7dc49`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:11.858506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:11.921187 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1401/2833] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:11.932418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:11.934149 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:11.990263 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd7dbb-ee3b-4948-8374-01b093b18852 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:11.993771 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ebbbb-ce85-4da5-91e8-2d8e1a068f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:11.994400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebbbb-ce85-4da5-91e8-2d8e1a068f75 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:11.994675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebbbb-ce85-4da5-91e8-2d8e1a068f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:11.994947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebbbb-ce85-4da5-91e8-2d8e1a068f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:12.004447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ebbbb-ce85-4da5-91e8-2d8e1a068f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:12.005878 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ebbbb-ce85-4da5-91e8-2d8e1a068f75 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:12.015024 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1433/2834] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:21.917984 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=34 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:22.010695 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:29.623435 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:29.686107 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163bc41-bc22-4309-a925-3ded487a85c5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:29.691383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:29.692029 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:29.692312 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:29.692584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] PATH_INFO: `/v3/users/76fc5d21a28a49b3896367cd2cd7dc49` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:29.693718 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76fc5d21a28a49b3896367cd2cd7dc49)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:29.695620 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:29.740979 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76fc5d21a28a49b3896367cd2cd7dc49'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:29.747814 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76fc5d21a28a49b3896367cd2cd7dc49'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:29.760382 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76fc5d21a28a49b3896367cd2cd7dc49'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:29.761019 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76fc5d21a28a49b3896367cd2cd7dc49'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:29.768105 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76fc5d21a28a49b3896367cd2cd7dc49'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:29.773871 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1402/2835] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:29 2022] DELETE /identity/v3/users/76fc5d21a28a49b3896367cd2cd7dc49 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:29.778331 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:29.778823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:29.778947 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:29.797394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:29.799113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:29.857749 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1434/2836] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:29.869574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:29.870995 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:29.927763 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3366ad0-a4a0-4915-841f-0c1ff90af663 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:29.931407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c7b8b-5a29-48d1-aaa3-9b3e9ec9a7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:29.932041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c7b8b-5a29-48d1-aaa3-9b3e9ec9a7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:29.932326 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c7b8b-5a29-48d1-aaa3-9b3e9ec9a7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:29.932600 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c7b8b-5a29-48d1-aaa3-9b3e9ec9a7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:29.941760 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c7b8b-5a29-48d1-aaa3-9b3e9ec9a7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:29.943178 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c7b8b-5a29-48d1-aaa3-9b3e9ec9a7f1 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:29.952584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1403/2837] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:30.222153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.223765 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:30.241990 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ee78-8aed-4856-83d0-f85db894d646 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.247564 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.248238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.248655 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.248793 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] PATH_INFO: `/v3/projects/96c05ee821094a619097e92d0ff108c5` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.249994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c05ee821094a619097e92d0ff108c5)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.251897 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.300000 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c05ee821094a619097e92d0ff108c5', 'request_id': 'req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:30.305902 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c05ee821094a619097e92d0ff108c5', 'request_id': 'req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:30.313057 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c05ee821094a619097e92d0ff108c5', 'request_id': 'req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:30.323009 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] The token cache is being invalidate because project 96c05ee821094a619097e92d0ff108c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:30.323291 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:30.323904 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-4569fdb8-cab5-40c0-9fa4-c9a6efc0f247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:30.325394 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1435/2838] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:30 2022] DELETE /identity/v3/projects/96c05ee821094a619097e92d0ff108c5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:30.331025 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.331195 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.331383 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.350879 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:30.352089 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:30.368123 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:30.411325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1404/2839] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:55:30.418752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.419035 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.419309 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.438238 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:30.439448 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:30.447138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1436/2840] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:30.451191 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.461559 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201969c-8fb6-43fb-9346-34248ef63a53 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.466017 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.466604 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.466878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.467144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.467778 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:55:30.468118 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.469698 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.481526 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1405/2841] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:55:30.485716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.494785 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d07f0-db94-4d50-9fef-2c12ecbeec0b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.499362 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.499970 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.500255 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.500524 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.501151 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.503150 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.532005 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1437/2842] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:55:30.536735 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.546605 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165680a-58e9-4b4d-98a3-e63d18d9ada6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.601710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.602248 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.602523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.602791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.603399 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.605311 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.625135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1406/2843] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:30.629850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.640266 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a91001-3ab7-4cd1-9e92-5e6ecaa29da9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.645546 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.646145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.646424 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.646698 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.647337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.648957 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.658788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1438/2844] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:30.662631 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.672550 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9188eda-e3e6-4aba-a2c3-729198b5d8d6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.677624 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f745a-079f-46c2-9ca8-53ef33b70f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.678242 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f745a-079f-46c2-9ca8-53ef33b70f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.678522 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f745a-079f-46c2-9ca8-53ef33b70f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.678791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f745a-079f-46c2-9ca8-53ef33b70f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.679427 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f745a-079f-46c2-9ca8-53ef33b70f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.681063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f745a-079f-46c2-9ca8-53ef33b70f12 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.692462 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1407/2845] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:30.696522 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.707010 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931d04-8f40-4afb-b229-5f998428b252 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.711701 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.712562 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.712847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.713129 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] PATH_INFO: `/v3/projects/3f966e4c453e45d6b82c99e0f32c8508/users/58f6d491cd5848a3be9f1df014956299/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.728136 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f966e4c453e45d6b82c99e0f32c8508, user_id=58f6d491cd5848a3be9f1df014956299, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.730766 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.744138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1439/2846] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:30 2022] PUT /identity/v3/projects/3f966e4c453e45d6b82c99e0f32c8508/users/58f6d491cd5848a3be9f1df014956299/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:30.755013 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.755295 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.755585 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.784272 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] MFA Rules not processed for user `58f6d491cd5848a3be9f1df014956299`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:30.785459 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:30.846858 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1408/2847] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:55:30.851432 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.872253 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:30.915395 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369710c-4d9b-49b3-a682-70259d405ac2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.920064 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:30.920663 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:30.920944 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:30.921215 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:30.921844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:30.924418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:30.950827 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1440/2848] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:30.955027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:30.965680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718a9f7-3c1f-42f2-811c-241e4e24ebf9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:30.978386 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:31.023501 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.024135 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.024412 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.024679 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.025299 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.027196 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.048162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1409/2849] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:30 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:31.052849 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.062865 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3804a-e693-4d0e-9f9e-fc49b3af497f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.067434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.068061 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.068337 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.068605 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.069237 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.070816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.081403 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1441/2850] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:31.085164 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.095226 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae335d6-1077-48ca-b980-cabbec89400a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.100574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b256b5b-fe62-4e66-9276-4647bb5d369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.101163 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b256b5b-fe62-4e66-9276-4647bb5d369f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.101441 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b256b5b-fe62-4e66-9276-4647bb5d369f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.101710 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b256b5b-fe62-4e66-9276-4647bb5d369f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.102350 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b256b5b-fe62-4e66-9276-4647bb5d369f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.103939 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b256b5b-fe62-4e66-9276-4647bb5d369f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.114588 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1410/2851] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:31.118490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.128616 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae49d3-76f1-438d-bfdb-944490b34a70 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.133082 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.133718 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.133994 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.134267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] PATH_INFO: `/v3/projects/b181743bf711413f9c526e1d60c53f73/users/1b81351750694eb494fae8c02a6bbd08/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.148886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b181743bf711413f9c526e1d60c53f73, user_id=1b81351750694eb494fae8c02a6bbd08, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.151656 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.166108 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1442/2852] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:31 2022] PUT /identity/v3/projects/b181743bf711413f9c526e1d60c53f73/users/1b81351750694eb494fae8c02a6bbd08/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:31.178067 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28061289-0288-4636-8511-eee1aa2a864a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.178353 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28061289-0288-4636-8511-eee1aa2a864a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.178629 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28061289-0288-4636-8511-eee1aa2a864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.206478 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-28061289-0288-4636-8511-eee1aa2a864a None None] MFA Rules not processed for user `1b81351750694eb494fae8c02a6bbd08`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:31.207676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-28061289-0288-4636-8511-eee1aa2a864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:31.272256 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1411/2853] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:31.277030 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.339020 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0987a0-3fee-42f4-a89d-4d80909fd76e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.343604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.344240 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.344516 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.344814 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.346063 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.348712 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.374622 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1443/2854] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:31 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:31.378816 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28061289-0288-4636-8511-eee1aa2a864a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.389929 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28061289-0288-4636-8511-eee1aa2a864a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.448154 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.448753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.449031 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.449303 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.449931 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.451864 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.473584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1412/2855] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:31 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:31.477672 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.488895 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221372-1e16-49e6-9ab5-452a505eeb47 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.494628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.495224 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.495506 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.495804 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.496460 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.498058 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.511185 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1444/2856] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:31.515928 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.526456 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a541ef6-06b4-49c8-baed-04d240f431d6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.531144 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.531781 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.532098 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.532379 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] PATH_INFO: `/v3/projects/72567e66978646119ae9d8e82274fec9/users/94625cdc9f804bf7a1a7a639ff88989e/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.545404 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:31.549128 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72567e66978646119ae9d8e82274fec9, user_id=94625cdc9f804bf7a1a7a639ff88989e, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.551930 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.565595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1413/2857] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:31 2022] PUT /identity/v3/projects/72567e66978646119ae9d8e82274fec9/users/94625cdc9f804bf7a1a7a639ff88989e/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:31.569929 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.581145 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b82598-d4e6-4bb6-a458-68dd3fda167c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.622144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:31.637418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4986aaf-2d25-426b-9a4d-0c0c567f33a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.637958 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4986aaf-2d25-426b-9a4d-0c0c567f33a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.638235 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4986aaf-2d25-426b-9a4d-0c0c567f33a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.638505 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4986aaf-2d25-426b-9a4d-0c0c567f33a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.639408 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4986aaf-2d25-426b-9a4d-0c0c567f33a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.640996 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4986aaf-2d25-426b-9a4d-0c0c567f33a9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.651435 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1445/2858] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:31.655756 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.667130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615ccc2-fdd4-4c59-a684-65741a7ae43e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.671714 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.672375 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.672652 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.672920 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] PATH_INFO: `/v3/projects/72567e66978646119ae9d8e82274fec9/users/94625cdc9f804bf7a1a7a639ff88989e/roles/580c9e29995a41c49a5a943d7b48532c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.674945 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72567e66978646119ae9d8e82274fec9, user_id=94625cdc9f804bf7a1a7a639ff88989e, role_id=580c9e29995a41c49a5a943d7b48532c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.677570 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.687704 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1414/2859] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:31 2022] PUT /identity/v3/projects/72567e66978646119ae9d8e82274fec9/users/94625cdc9f804bf7a1a7a639ff88989e/roles/580c9e29995a41c49a5a943d7b48532c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:31.697978 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.698261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.698536 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.727335 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] MFA Rules not processed for user `94625cdc9f804bf7a1a7a639ff88989e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:31.728575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:31.796116 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1446/2860] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:31.807443 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:31.809189 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:31.875229 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89572af6-9ff3-40b9-a6c1-3f1d0c5fd7b7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:31.880513 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:31.881124 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:31.881400 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:31.881672 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:31.938696 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:31.940574 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:31.949657 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1415/2861] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:33.444884 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:33.446273 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:33.458900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f4a43-0dc7-40fa-b2eb-a73192c5b6b9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:33.462446 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:33.463043 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:33.463322 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:33.463597 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:33.524894 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:33.526319 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:33.536111 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1447/2862] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:34.056790 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:34.065328 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4422c3a-8570-41a2-af1b-0e2f4b97439b service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:34.068944 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca48e02-5d8e-4bda-ba51-abf359721bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:34.069539 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca48e02-5d8e-4bda-ba51-abf359721bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:34.069846 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca48e02-5d8e-4bda-ba51-abf359721bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:34.070122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca48e02-5d8e-4bda-ba51-abf359721bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:34.078973 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca48e02-5d8e-4bda-ba51-abf359721bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:34.081000 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca48e02-5d8e-4bda-ba51-abf359721bd5 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:34.090407 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1416/2863] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:37.965138 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:38.027545 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efd366-b711-4e9f-b403-58895d16a047 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:38.032113 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:38.032739 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:38.033553 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:38.033934 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] PATH_INFO: `/v3/users/58f6d491cd5848a3be9f1df014956299` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:38.035067 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f6d491cd5848a3be9f1df014956299)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:38.037008 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:38.041761 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:38.085980 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f6d491cd5848a3be9f1df014956299'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.092807 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f6d491cd5848a3be9f1df014956299'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.093468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f6d491cd5848a3be9f1df014956299'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.101489 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f6d491cd5848a3be9f1df014956299'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.114451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f6d491cd5848a3be9f1df014956299'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.120327 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1448/2864] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:37 2022] DELETE /identity/v3/users/58f6d491cd5848a3be9f1df014956299 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:38.124784 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:38.125268 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:38.125387 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:38.145122 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:38.146254 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:38.206523 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1417/2865] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:38.216410 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:38.217826 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:38.276580 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb335de-dcf0-4c7b-9f74-0edbba7d45fb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:38.280380 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:38.280979 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:38.281261 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:38.281530 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:38.291168 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:38.292654 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:38.301928 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1449/2866] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:38.574380 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:38.576302 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:38.589230 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f049-277e-4949-8ca3-57f8cf193789 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:38.593867 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:38.594502 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:38.594811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:38.595060 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] PATH_INFO: `/v3/projects/3f966e4c453e45d6b82c99e0f32c8508` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:38.596296 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f966e4c453e45d6b82c99e0f32c8508)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:38.598206 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:38.614734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:38.649183 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f966e4c453e45d6b82c99e0f32c8508', 'request_id': 'req-ae006088-981e-4284-bd8e-38a814fa8ddb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.655136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f966e4c453e45d6b82c99e0f32c8508', 'request_id': 'req-ae006088-981e-4284-bd8e-38a814fa8ddb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.660772 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f966e4c453e45d6b82c99e0f32c8508', 'request_id': 'req-ae006088-981e-4284-bd8e-38a814fa8ddb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.672667 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] The token cache is being invalidate because project 3f966e4c453e45d6b82c99e0f32c8508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:38.672951 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.673641 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.675215 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1418/2867] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:38 2022] DELETE /identity/v3/projects/3f966e4c453e45d6b82c99e0f32c8508 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:38.679115 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:38.688510 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c700-9ebb-4d26-b311-6c27c8e4c12b service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:38.746075 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:38.746704 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:38.746988 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:38.747267 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] PATH_INFO: `/v3/users/1b81351750694eb494fae8c02a6bbd08` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:38.748421 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b81351750694eb494fae8c02a6bbd08)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:38.750334 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:38.780375 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:38.800593 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b81351750694eb494fae8c02a6bbd08'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.807167 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b81351750694eb494fae8c02a6bbd08'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.807816 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b81351750694eb494fae8c02a6bbd08'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.815612 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b81351750694eb494fae8c02a6bbd08'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.827710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b81351750694eb494fae8c02a6bbd08'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:38.834259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1450/2868] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:38 2022] DELETE /identity/v3/users/1b81351750694eb494fae8c02a6bbd08 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:39.104682 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:39.106168 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:39.121030 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae006088-981e-4284-bd8e-38a814fa8ddb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:39.182438 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:39.183120 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:39.183250 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:39.183511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] PATH_INFO: `/v3/projects/b181743bf711413f9c526e1d60c53f73` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:39.184675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b181743bf711413f9c526e1d60c53f73)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:39.186519 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:39.232071 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b181743bf711413f9c526e1d60c53f73', 'request_id': 'req-3c75e51e-4421-43be-bf07-145ca1d7219a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.238841 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b181743bf711413f9c526e1d60c53f73', 'request_id': 'req-3c75e51e-4421-43be-bf07-145ca1d7219a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.244338 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b181743bf711413f9c526e1d60c53f73', 'request_id': 'req-3c75e51e-4421-43be-bf07-145ca1d7219a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.257026 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] The token cache is being invalidate because project b181743bf711413f9c526e1d60c53f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:39.257315 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.257948 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.259419 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1419/2869] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:39 2022] DELETE /identity/v3/projects/b181743bf711413f9c526e1d60c53f73 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:39.263487 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:39.265180 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:39.277883 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5721104-158d-4d91-ac9e-25958cd0b589 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:39.333716 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:39.334339 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:39.334617 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:39.334889 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] PATH_INFO: `/v3/users/94625cdc9f804bf7a1a7a639ff88989e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:39.336015 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] RBAC: Authorizing `identity:delete_user(user_id=94625cdc9f804bf7a1a7a639ff88989e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:39.337906 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:39.384966 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94625cdc9f804bf7a1a7a639ff88989e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.392324 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94625cdc9f804bf7a1a7a639ff88989e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.393014 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94625cdc9f804bf7a1a7a639ff88989e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.400473 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94625cdc9f804bf7a1a7a639ff88989e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.413044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-333742fa-b976-4d1e-9328-ace4dd36391b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94625cdc9f804bf7a1a7a639ff88989e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.419044 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1451/2870] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:39 2022] DELETE /identity/v3/users/94625cdc9f804bf7a1a7a639ff88989e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:39.711440 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:39.712870 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:39.726961 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75e51e-4421-43be-bf07-145ca1d7219a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:39.758543 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:39.788021 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:39.788656 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:39.788936 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:39.789211 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] PATH_INFO: `/v3/projects/72567e66978646119ae9d8e82274fec9` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:39.790334 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72567e66978646119ae9d8e82274fec9)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:39.792287 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:39.839450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72567e66978646119ae9d8e82274fec9', 'request_id': 'req-9465a5b5-c9ed-4175-a275-8b1b46636ea4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.846160 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72567e66978646119ae9d8e82274fec9', 'request_id': 'req-9465a5b5-c9ed-4175-a275-8b1b46636ea4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.851827 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72567e66978646119ae9d8e82274fec9', 'request_id': 'req-9465a5b5-c9ed-4175-a275-8b1b46636ea4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.866579 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] The token cache is being invalidate because project 72567e66978646119ae9d8e82274fec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:39.867066 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.868174 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9465a5b5-c9ed-4175-a275-8b1b46636ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:39.869958 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1420/2871] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:39 2022] DELETE /identity/v3/projects/72567e66978646119ae9d8e82274fec9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:39.874903 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:39.875191 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:39.875470 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:39.900090 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:39.901314 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:39.920382 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:39.963485 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1452/2872] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:39.972012 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:39.972306 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:39.972584 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:39.992175 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:39.993378 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:40.002148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1421/2873] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:40.006528 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.016680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f8a0a-211f-482f-bc77-9d9871ab6d48 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.022054 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.022688 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.022969 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.023245 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.023891 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:55:40.024241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.025829 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.039615 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1453/2874] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:55:40.045450 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.054811 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168378-e6f2-4d9c-a736-401c41ab52b9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.059457 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.060094 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.060374 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.060645 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.061269 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.063151 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.091819 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1422/2875] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:40 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:40.096096 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.106172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd267-f90a-4f22-bf37-335dae25ddb0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.162614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.163213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.163490 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.163764 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.164413 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.166300 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.186611 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1454/2876] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:40 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:40.191472 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.201556 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76b567-a9ac-49e0-9587-e44afc0d1c08 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.207102 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.207688 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.207988 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.208263 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.208903 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.210476 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.220511 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1423/2877] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:40.224415 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.234223 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e0ea6-c468-4c1d-b8ec-fd9a16eedee9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.239391 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e40e85-d69a-4b2e-9707-245d5219d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.240021 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40e85-d69a-4b2e-9707-245d5219d993 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.240317 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40e85-d69a-4b2e-9707-245d5219d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.240575 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40e85-d69a-4b2e-9707-245d5219d993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.241243 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e40e85-d69a-4b2e-9707-245d5219d993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.242860 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e40e85-d69a-4b2e-9707-245d5219d993 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.254409 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1455/2878] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:40.258548 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.269236 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e585a-aa68-4c23-b375-d78ba192a7be admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.273782 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.274411 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.274685 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.274953 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] PATH_INFO: `/v3/projects/a240aaec5b7e4a3880a3c2c155286dda/users/3965440563bd45dfa821a42a2484a276/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.289414 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a240aaec5b7e4a3880a3c2c155286dda, user_id=3965440563bd45dfa821a42a2484a276, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.292056 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.304989 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1424/2879] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:40 2022] PUT /identity/v3/projects/a240aaec5b7e4a3880a3c2c155286dda/users/3965440563bd45dfa821a42a2484a276/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:55:40.316046 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.316526 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.316628 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.345326 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] MFA Rules not processed for user `3965440563bd45dfa821a42a2484a276`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:40.346518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:40.408264 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1456/2880] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:40.419363 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.421052 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:40.476162 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d76c2-71bf-4f0d-83a1-1fba5b881500 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.479651 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a2146-3f0e-4ab6-a53e-629563661493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.480277 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a2146-3f0e-4ab6-a53e-629563661493 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.480552 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a2146-3f0e-4ab6-a53e-629563661493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.480822 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a2146-3f0e-4ab6-a53e-629563661493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.489673 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a2146-3f0e-4ab6-a53e-629563661493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.491085 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a2146-3f0e-4ab6-a53e-629563661493 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.499642 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1425/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:40.524710 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.545106 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:40.588172 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0e797-5b5c-4be4-a9c1-c56af5ee5367 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.593330 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.593963 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.594244 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.594520 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] PATH_INFO: `/v3/users/3965440563bd45dfa821a42a2484a276` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.595819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3965440563bd45dfa821a42a2484a276)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.597768 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.643604 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3965440563bd45dfa821a42a2484a276'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:40.650352 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3965440563bd45dfa821a42a2484a276'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:40.651026 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3965440563bd45dfa821a42a2484a276'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:40.657745 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3965440563bd45dfa821a42a2484a276'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:40.669270 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3965440563bd45dfa821a42a2484a276'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:40.675496 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1457/2882] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:40 2022] DELETE /identity/v3/users/3965440563bd45dfa821a42a2484a276 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:40.680087 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.680436 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.680734 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.687677 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:40.701695 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:40.702878 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:40.760872 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1426/2883] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:55:40.770819 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:40.772260 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:40.829587 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797cac5-80df-4489-8443-33793519f398 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:40.833447 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bb289-adb3-44ab-9f26-9161f949774f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:40.834050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bb289-adb3-44ab-9f26-9161f949774f service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:40.834332 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bb289-adb3-44ab-9f26-9161f949774f service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:40.834608 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bb289-adb3-44ab-9f26-9161f949774f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:40.843437 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bb289-adb3-44ab-9f26-9161f949774f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:40.844885 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bb289-adb3-44ab-9f26-9161f949774f service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:40.854846 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1458/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:55:41.133977 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.135562 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:41.149309 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeecc39-ce74-4cad-8531-0424087f5d8b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.154270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.155621 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.155911 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.156325 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] PATH_INFO: `/v3/projects/a240aaec5b7e4a3880a3c2c155286dda` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.157416 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a240aaec5b7e4a3880a3c2c155286dda)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.160047 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.206225 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a240aaec5b7e4a3880a3c2c155286dda', 'request_id': 'req-a8b9a459-48d1-46cb-9701-a8062e8821f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:41.212247 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a240aaec5b7e4a3880a3c2c155286dda', 'request_id': 'req-a8b9a459-48d1-46cb-9701-a8062e8821f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:41.218361 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a240aaec5b7e4a3880a3c2c155286dda', 'request_id': 'req-a8b9a459-48d1-46cb-9701-a8062e8821f1'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:41.231489 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] The token cache is being invalidate because project a240aaec5b7e4a3880a3c2c155286dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:41.231777 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:41.232506 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a8b9a459-48d1-46cb-9701-a8062e8821f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:41.234027 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1427/2885] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:41 2022] DELETE /identity/v3/projects/a240aaec5b7e4a3880a3c2c155286dda => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:41.238886 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.239259 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.239541 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.258752 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:41.260074 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:41.299533 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:41.321479 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1459/2886] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:41.329408 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.330184 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.330458 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.351453 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:41.352675 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:41.360488 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1428/2887] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:41.364828 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.374400 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9cdfa-4af6-40d4-946a-ff0bfb3e60f4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.378926 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.379518 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.379795 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.380105 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.380741 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:55:41.381041 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.382614 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.394922 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1460/2888] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 16:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:55:41.399014 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.408921 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab889c5-5a97-433d-81cc-cd4c5450407b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.413512 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.414097 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.414368 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.414637 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.415241 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.417136 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.444207 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1429/2889] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 16:55:41 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:55:41.448451 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.459371 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d52f-3c67-4509-9b56-b3f6e0edd319 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.515418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.516048 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.516343 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.516603 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.517863 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.520144 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.540157 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1461/2890] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 16:55:41 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:55:41.544218 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.554661 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb139e-4c5c-4442-aa00-592612979ec1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.559130 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.559730 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.560041 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.560318 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.560964 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.562547 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.576680 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1430/2891] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:55:41.581805 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.592134 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010609f3-e4c4-4b25-9b87-6d4931066d15 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.596847 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352e2c5-e90b-4ff8-96c1-fb72d5568ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.597439 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352e2c5-e90b-4ff8-96c1-fb72d5568ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.597714 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352e2c5-e90b-4ff8-96c1-fb72d5568ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.597987 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352e2c5-e90b-4ff8-96c1-fb72d5568ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.598621 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352e2c5-e90b-4ff8-96c1-fb72d5568ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.600230 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352e2c5-e90b-4ff8-96c1-fb72d5568ccb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.610574 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1462/2892] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 16:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:55:41.614660 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.624676 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d447b-4a68-4b15-bc9d-9d738ece7cdd admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.630791 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.631433 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.631713 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.632043 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] PATH_INFO: `/v3/projects/fceef518f3ab4830b7c23277d3039afa/users/c705b566828b41079820df26dbeb426f/roles/e1f816c1dc84469786d5a59d7920b1f1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.647186 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fceef518f3ab4830b7c23277d3039afa, user_id=c705b566828b41079820df26dbeb426f, role_id=e1f816c1dc84469786d5a59d7920b1f1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.649856 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.662077 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1431/2893] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 16:55:41 2022] PUT /identity/v3/projects/fceef518f3ab4830b7c23277d3039afa/users/c705b566828b41079820df26dbeb426f/roles/e1f816c1dc84469786d5a59d7920b1f1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:41.673241 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.673548 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.673892 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.701464 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] MFA Rules not processed for user `c705b566828b41079820df26dbeb426f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:41.702680 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:41.766069 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1463/2894] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:41.778155 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:41.779884 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:41.806132 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:41.839840 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf867d1-c914-4afd-8385-9f6927abca2a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:41.843437 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2336b-5143-46be-a06e-994e42bdc932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11df13be0c5543748f98b9ea3d83c3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:41.844062 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2336b-5143-46be-a06e-994e42bdc932 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:41.844340 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2336b-5143-46be-a06e-994e42bdc932 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:41.844608 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2336b-5143-46be-a06e-994e42bdc932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:41.853336 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2336b-5143-46be-a06e-994e42bdc932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:41.854753 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2336b-5143-46be-a06e-994e42bdc932 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:41.864322 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1432/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:48.919236 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:48.960465 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:48.983467 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea5e3-77bf-4c9e-be95-b3cebbd1c754 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:48.988154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:48.988788 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:48.989073 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:48.989357 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] PATH_INFO: `/v3/users/c705b566828b41079820df26dbeb426f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:48.990491 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c705b566828b41079820df26dbeb426f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:48.992444 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:49.043468 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c705b566828b41079820df26dbeb426f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.050213 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c705b566828b41079820df26dbeb426f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.050868 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c705b566828b41079820df26dbeb426f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.058418 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c705b566828b41079820df26dbeb426f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.070154 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c705b566828b41079820df26dbeb426f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.075900 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1464/2896] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 16:55:48 2022] DELETE /identity/v3/users/c705b566828b41079820df26dbeb426f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:49.080475 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:49.081589 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:49.082024 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:49.101463 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] MFA Rules not processed for user `bcc81a9e50c0416682e495a7f72fd265`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:49.102654 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:49.162148 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1433/2897] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 16:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:49.171844 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:49.173321 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:49.231023 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dcff0-59a6-4d72-9f41-978206858633 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:49.234675 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '505ea6d7aac346e1a604b192a2a0c364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:49.235271 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:49.235550 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:49.235823 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:49.245050 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:49.246486 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:49.256434 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1465/2898] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:49.524857 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:49.526317 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:49.540270 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51d7f5-0f50-4410-967e-a373385a7b4d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:49.544999 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcc81a9e50c0416682e495a7f72fd265', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad742c0425e4471ab69af90dda50deea', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:49.545779 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:49.546063 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:49.546341 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] PATH_INFO: `/v3/projects/fceef518f3ab4830b7c23277d3039afa` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:49.547480 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fceef518f3ab4830b7c23277d3039afa)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:49.549418 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:49.596839 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fceef518f3ab4830b7c23277d3039afa', 'request_id': 'req-e5066b99-53ed-496b-b812-e5e45c75c7ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.602901 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fceef518f3ab4830b7c23277d3039afa', 'request_id': 'req-e5066b99-53ed-496b-b812-e5e45c75c7ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.608595 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fceef518f3ab4830b7c23277d3039afa', 'request_id': 'req-e5066b99-53ed-496b-b812-e5e45c75c7ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.623731 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] The token cache is being invalidate because project fceef518f3ab4830b7c23277d3039afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:55:49.624058 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.624709 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e5066b99-53ed-496b-b812-e5e45c75c7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:55:49.626199 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1434/2899] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 16:55:49 2022] DELETE /identity/v3/projects/fceef518f3ab4830b7c23277d3039afa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:59.251632 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=62,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:59.620316 dv-cinder-779578-5-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:38.166961 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:38.168411 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:38.301558 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b73e7a-4c54-4967-932e-483306eb0043 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:38.359219 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e605979196e14110a19a6aff30f2006d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77eff9a0065049a29630adf71df889d3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:38.359850 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:38.360153 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:38.360426 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:38.362110 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:38.374360 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:38.375920 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7223b-5b02-43b4-b4b8-462d35d7eb72 service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:38.385770 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1466/2900] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:48.381449 dv-cinder-779578-5-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}